Image coding and decoding device

ABSTRACT

In an image coding and decoding device, input digital image signals are divided into blocks of prescribed size, and coding processing is performed on differential signals taken between the input block signal and interframe forcasting signals thereby enabling efficient transmission to be performed.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image coding and decoding device,and more particularly to an image coding and decoding device which canefficiently code and decode motion image signals so as to transmit andreceive the signals by the interframe coding.

2. Description of the Prior Art

FIG. 3 is a block diagram which shows a transmitting section of an imagecoding transmitter in the prior art. In FIG. 3, numeral 2 designates amotion compensating circuit, numeral 6 designates a differentialsection, numeral 8 designates a coding and decoding circuit, numeral 11designates an adder, and numeral 13 designates a frame memory.

FIG. 4 is a block diagram which shows a motion vector detecting sectionin the motion compensating circuit 2. In FIG. 4, numeral 14 designates adifferential section, numeral 16 designates a distortion operator, andnumeral 18 designates a comparator.

The operation of the device will be described.

In the motion compensating circuit 2, using digitally transformed inputsignals 1 and previous frame reconstruction signals 3 stored in theframe memory 13, with the picture element blocks of the prescribed sizeN₁ ×N₂ (N₁, N₂ being plus integers) used as a unit, a block which hasthe pattern most similar to that of the block of the input signal isdetected from the previous signals, and signals of the detected blockare outputted as predicting signals 4, and index information 5indicating motion vectors being position shift between the position ofthe block and that of the input signal block is transmitted to thereceiving side.

An example of the motion detecting method is shown in FIG. 4.

Assuming that the input signal block is S(1), the previous frame signalblock is S'_(i) (3), the number of the searched blocks is L (L beingplus integer), the matching distortion between the blocks be d_(i) (17),and K=N₁ ×N₂ ##EQU1## Among d_(i) (11) estimated by the differentialsection 14 and the distortion operator 16 based on the above formulas,the block having the smallest matching distortion value is detected bythe comparator 18, and the block signal is transformed into thepredicting signal 4 block by block, and the index information 5 of theblock motion vectors is outputted. However, in FIG. 4, the hardwarecomposition is simplified by applying the serial process to thedistortion operation and the comparing process at each searching block.In FIG. 5, an example of the arrangement of the searching vectors isshown.

Differences between the predicting signals 4 estimated as abovedescribed and the input signals 1 are taken, and the differentialsignals 7 are coded by the coding and decoding circuit 8, and then thecoded information 9 is transmitted to the receiving side and at the sametime the differential decoded signals 10 by decoding the codedinformation and the predicting signals 4 are added and thereby thereconstructed signals 12 are obtained. After one frame of thesereconstructed signals 12 is stored in the frame memory 13, they are readout as the previous frame reconstructed signals 3 at the next coding.

In the coding and decoding circuit 8, coding is executed only when thevalue of the differential signals 7 is larger than a prescribedthreshold value, and the picture element supplementing (the value of thepredicting signals 4 can be made equal to that of the reconstructedsignals 12 by making naught the differential reconstructed signalswithout coding) is performed in other cases; thereby the generatinginformation content can be restrained.

Since the motion compensating device in the prior art is constituted asabove described, the searched vectors are limited in number, resultingin the most suitable motion compensation being made impossible becauseof the image being out of the searched range in case its movement is tooextensive. Furthermore there is another problem in that in case of thedeterioration of the picture quality, or in case of the changing ofscenes, the matching precision between frames is low, and the restrainteffect of the predicting error signal component does not appear, causingan increase of the generating information and deterioration of thepicture quality.

FIGS. 10 and 11 are block diagrams which show the coding section of theconventional image signal progressive build-up coding and decodingdevice, and the composition model of the decoding section, which are,for example, disclosed in the column "Vector Quantization of the ImageSignal" of the Journal of the Institute of Television Engineers ofJapan, Vol. 38, No. 5, pp. 452˜457 (1984). In the coding section in FIG.10, numeral 101 designates the input signal vectors which block theinput image signal system of the static image data by the unit of thesample m×n (m, n being integers), numeral 102 designates a subtractorwhich finds the residual signal vectors 103 between the input signalvectors the previous stage decoded vectors delayed by one frame, numeral128 designates an average value separated and normalized vectorquantization decoder, numeral 110 designates the amplitude coded data,numeral 129 designates the amplitude coded data, numeral 130 designatesthe output vector index coded data, numeral 114 designates a codeassignment circuit, numeral 115 designates the coding device outputsignals, numeral 117 designates the residual signal decoded vectors,numeral 118 designates an adder which adds the residual signal decodedvectors to the front stage decoded vectors delayed by one frame, numeral119 designates the decoded vectors, and numeral 120 designates a framememory for delaying the decoded vectors by one frame.

In the decoding section in FIG. 11, numeral 115 designates the codingdevice output signals obtained in the code section, numeral 124designates a coding assignment circuit which inverts the codingassignment, and numeral 131 designates an average value separated andnormalized vector quantization decoder which carries out the averagevalue separated and normalized vector quantization decoding.

First, the principle of the vector quantization will be describedbriefly. The input signal systems of K pieces as a whole shall be madeinput vectors x={x₁, x₂, . . . ,x_(k) }. Then a set of N pieces of therepresentative points (that is, the output vectors) y_(i) ={y_(i1),y_(i2), . . . , Y_(ik) } of K-dimensional Euclid signal space R^(K)(xεR^(K)) shall be made Y=[y₁, y₂, . . . , y_(N) ]. The vector quantizersearches the output vector y_(i) which is in the shortest distance fromthe output vectors, and it defines the output vector as follows:

    if d(x, y.sub.i)<d(x, y.sub.l) for all l x→y.sub.i

Where d(x, y_(i)) indicates the distance between the input and outputvectors (distortion). Then the input vector x is transmitted or recordedby the index i of the output vectors, and substituted by y_(i) at thereconstruction state.

The average value separated and normalized vector quantization suppliesthe average value separated and normalized of the input vectors of thevector quantizer so as to limit the distribution of the output vectorson the superunit spherical surface in the multidimensional signal space.Let the input signal vectors be S={S₁, S₂, . . . , S_(K) }, and theaverage value μ, the amplitude σ, and the average value separated andnormalized vectors x are represented respectively as follows: ##EQU2##Where as the approximate formulas of the amplitude σ, following formulascan be used. ##EQU3##

The scalar quantization of the average value μ and the amplitude σ, andthe vector quantization of the average value separate input vectors xcan make the generality of the vector quantizer with the limited numberof output vectors much wider so as to develop its coding performance.

The operation of the image signal progressive build-up decoder in theprior art will be described. In FIG. 10, at the first stage of coding,the frame memory 120 which stores the prestage decoded results is leftcleared, and the input signal vectors 101 pass through the subtractor102 as they are, and the average value separated and normalized vectorquantization is performed as the residual signal vectors 103 in theaverage value separated and normalized vector quantizer 128, thereby thecoding data 110 of the average value, the coding data 129 of theamplitude, the coding data 130 of the output vector index, and theresidual signal decoded vectors 117 are outputted. Each of the codeddata 110, 129, 130 is converted into the suitable code words by the codeassignment circuit 114 and then transmitted. The residual signal decodedvectors 117 pass through the adder 118, as they are, and are written asthe decoded vectors 119 to the frame memory 120. At the second codingstage, in the subtractor 102 the residual signal vectors 103 areobtained by subtracting the previous stage decoded vectors from theinput signal vectors 101. The residual signal vectors 103 are quantizedinto the average value separated and normalized vectors in the averagevalue separated and normalized quantizer 128, thereby the coding data110 of the average value, the coded data 129 of the amplitude, the codeddata 130 of the output vector index, and the residual signal decodedvectors 117 are outputted. Each of the coded data 110, 129, 130 isconverted into the suitable code words in the code assignment circuit114 and then transmitted. The residual signal decoded vectors 117 areadded to the front stage decoded vectors in the adder 118 so as to renewthe frame memory 120 as the decoded vectors 119. These processes arefollowed by the same coding operations repeated at each stage so as totransmit each of the coded data 110, 129, 130, resulting in renewing theframe memory 120.

In FIG. 11, when the decoding starts, the frame memory 120 which storesthe previous stage decoded results is left cleared. The coding deviceoutput signals 115 obtained in the coding section are inverted withrespect to the code assignment in the code assignment circuit 124 so asto produce each of the coded data 110, 129, 130. Each of the coded data110, 129, 130 is inputted into the average separated and normalizedvector quantization decoder 131, where the residual signal decodedvectors are obtained from the decoded average value and the amplitudereconstructed vectors. In the adder 118, the residual signal decodedvectors 117 are added to the front stage decoded results of the framememory 120 so as to renew the frame memory 120. These operations arerepeated at each stage. Since the image signal progressivereconstruction coding device in the prior art is composed as abovedescribed, it is difficult to control the amount of the codedinformation generation and the reconstructed image quality suitably inwider range.

FIG. 16 is a block diagram which shows a composition example of atransmission section of an interframe coding device disclosed, forexample, in the Technical Report of the Institute of Electronics andCommunication Engineers of Japan IE84-1- ('84). In FIG. 16, numeral 201designates the digitalized motion image signal system, numeral 207designates a subtractor for obtaining the interframe differentialsignals, numeral 208 designates the interframe predicting signals formedfrom the past time frame which already finished coding, numeral 209designates the interframe differential signal system, numeral 210designates a block discrimination section which quantizes the interframedifferential signals 209 of the predicting error to naught, anddiscriminates block by block whether the coding at the next stagenecessary or not, numeral 211 designates the threshold value for theblock discrimination, numeral 212 a coding control section which decidesthe threshold value 211, numeral 213 designates a coding and decodingsection which encodes and decodes the blocks judged as the significantpredicting errors (they are called as the significant blocks and theblocks whose predicting errors are regarded as naught are calledineffective blocks) in the block discriminating section 210, numeral 214designates the decoded predicting error signals obtained in the codingand decoding section 213, numeral 215 designates an adder which adds thepredicting signals 208 to the decoded predicting error signals 214,numeral 216 designates the decoded image signal series, numeral 217designates a frame memory which forms the interframe predicting signals208 using the decoded image signal series 216, numeral 218 designatesthe coded data obtained in the coding and decoding section 213, numeral219 designates a variable length coding section which encodes the codeddata 218 in variable length, numeral 220 designates a buffer forsmoothing where the variable length coded data processed in the variablelength coding section 219 is transmitted at the constant transmittingspeed, numeral 221 designates the data that count the variable lengthcoded data series per frame (the amount of the generated information perframe), numeral 222 designates a line interface (I/F) section fortransmitting the smoothed data system by the buffer 220 into thetransmission line, and numeral 223 designates the transmission signals.FIG. 17 is a block diagram showing a composition example of the codingcontrol section 212 in FIG. 16. In FIG. 17, numeral 232 designates aregister for delaying the threshold value 211 in frame period, numeral233 designates the threshold value delayed by the register 232, andnumeral 234 designates a table ROM. FIG. 18 is a diagram which explainsthe characteristics to be written in the table RAM 234.

The operation of the device will be described. The digitalized imagesignal series 201 is converted into the interframe differential signals209 through the subtraction between the interframe predicting signal 208by the subtractor 207. The interframe predicting signals 208 are formedusing the reconstruction image signal system of the past time framewhich has finished coding and local decoding in the transmissionsection; therein some other methods like the motion compensation aresometimes applied. The interframe differential signals 209 come near tonaught in the case of no change or movement in a subject, and the blockdiscrimination section 210 discriminates the significant blocks or theinsignificant blocks so as to transmit only the data relating to theinsignificant blocks resulting in compressing the amount of theinformation. In order to discriminate blocks, the sum of the absolutevalue of the blocked interframe differential signal system is found andcompared with the threshold value 211. Let the blocked interframedifferential signal system be ε_(j) (j=1, 2, . . . , k) and thethreshold value be T_(n), then the discrimination of the blocks isexecuted as follows: ##EQU4## Where with respect to the blocksrecognized as significant, the interframe differential signals 209 areencoded. The coding and decoding section 213 encodes the significantblocks and then decodes them so as to output the decoded predictingerror signals 214 and the coded data 218. There are various coding anddecoding methods, but they have little relationship to the invention andthe detailed description shall be omitted here. The decoded predictingerror signals 214 are added by the adder 215 to the interframepredicting signals 208 so as to form the decoded image signal series216. The decoded image signal series 216 is stored in the frame memory217, and used for forming the interframe predicting signals in the nextframe and so forth. On the other hand, the coded words are assigned tothe coded data 218 in the variable length coding section 219corresponding to generation frequency of each data. The buffer 220smooths the speed so as to transmit the variable length coding datasystem at the constant transmitting speed, and counts the variablelength coded data series per frame and outputs them as the informationamount 221 per frame to the coding control section 212. The lineinterface section 222 transmits the speed smoothed variable length codeddata as the transmission signals 223 into the transmission line.

The coding control section 212, referring to the table ROM 234 by usingthe threshold value 221 given the frame delay by the register 232 andthe amount of information, outputs the new threshold value 211. Let theprevious threshold value 233 be T_(n-1) and the information amount 221in the encoded frame using this T_(n-1) be B_(n-1), then the newthreshold value T_(n) (211) is obtained as shown in FIG. 18. Between thethreshold value and the information amount, there is a hyperbolicrelation as shown in FIG. 18. In FIG. 18, there are four curves written,which vary according to the width of the movement of a subject. WhenB_(n-1) and T_(n-1) are given, the most suitable curve is chosen amongthe plural curves stored in the table ROM 234. Then the point B* of theamount of information is looked for along the chosen curve where B* isthe information amount admitted to one frame in accordance with thetransmission speed. When the point of the amount of information becomingB* is decided on the chosen curve, the threshold value at the point isread as T_(n).

In other words, according to the relation between the threshold valueT_(n-1) of the frame which finishes the coding and the informationamount B_(n-1), the characteristic curve corresponding to the amount ofmotion is decided and controlled so as to obtain the necessary thresholdvalue to attain the aimed information amount as the new threshold valueT_(n) from the curve.

Since the interframe coding device is constituted as above described,delay is always produced in controlling the threshold value forsmoothing the amount of the information. Accordingly it isdisadvantageous in that when the subject is transferred from the staticstate to the dynamic state or from the dynamic state to the staticstate, an extreme time lapse may be produced or the focusing of thepicture quality is likely to be delayed.

FIG. 23 is a block diagram showing a composition example of atransmission section of an image signal progressive reconstructioncoding device in the prior art using the vector quantizer, which isdisclosed, for example, in the Technical Report of the Institute ofElectronics and Communication Engineers of Japan IT85-61 (1985) titling"Image High Performance Coding by Vector Quantization".

In FIG. 23, numeral 301 designates the input signal vectors which blockthe input signal system of the static image data and so on by everysample m×n (m, n being natural numbers), numeral 302 designates asubtractor which obtains the residual signal vectors of the input signalvectors 301 and the previous stage decoded vectors 319 delayed by oneframe, numeral 303 designates the residual signal vectors, numeral 306designates an average value coding and decoding device which finds theaverage value within block at each vector and gives the high performancecoding and decoding to the average value and outputs the average valuecoded data 308 and the average value decoded value 309, numeral 307designates a normalized vector quantization coding and decoding devicewhich converts the residual signal vector 303 into the normalized outputvectors and the amplitude gain through the inner product vectorquantizer and outputs the vector quantization coded data 310 and theamplitude reconstructed output vector 311, numeral 308 designates theaverage value coded data, numeral 309 designates the average datadecoded value, numeral 310 designates the vector quantization codeddata, numeral 311 designates the amplitude reconstructed vector, numeral312 designates a first adder which adds the average value decoded datato the amplitude reconstructed vector so as to find the residual signaldecoded vector 315, numeral 315 designates the residual signal decodedvector, numeral 316 designates a second adder which adds the residualsignal decoded vector 315 to the front stage decoded vector 319 delayedby one frame so as to obtain the decoded vector 317 at each stage,numeral 317 designates the decoded vector at each stage, numeral 318designates a frame memory which delays the decoded vector by one frame,numeral 319 designates the previous stage decoded vector outputted fromthe frame memory 318, numeral 320 designates a code assignment circuitwhich transforms the average value coded data 308 and the vectorquantization data 310 into the coded words using the variable lengthcoding and then outputs the coded words, and numeral 324 designates thecoded output data.

The operation of the device will be described. In FIG. 23, at the timeof the first stage coding, the frame memory 318 is left cleared. Theinput signal vectors 301 are not at all processed in the subtractor 302,but outputted as the residual signal vectors 303 as they are. In thenormalized vector quantization coding and decoding device 307, theaverage value of the residual signal vectors 303 is estimated, and thenthe average value is subjected to the high performance coding using theDPCM coding method or the like so as to output the average value codeddata 308 and the average value decoded data 309 locally decoded. On theother hand, with respect to the normal vector quantization, the residualsignal vectors 303 are converted through the inner vector quantizer intothe normalized output vectors and the amplitude gain, and the vectorquantization coded data 310 and the amplitude reconstructed outputvectors 311 are outputted.

The operation principle of the inner product vector quantizer will bedescribed referring to FIG. 24. In the inner product vector quantizer,the average value separation and normalization is given, and a set ofthe normalized output vector y_(i) arranged on the unit supersphericalsurface of the multidimensional signal space, Y=[y₁, y₂, . . . , y_(N) ]is used. That is, the normalized output vector y_(i) satisfies thefollowing conditions simultaneously. ##EQU5##

The input vector x to the inner product vector quantizer is convertedthrough the operation process of the following formulas into thenormalized output vector y_(i) which gives the maximum inner productvalue to the input vector x, and the maximum inner product value isgiven as the amplitude gain g of the input vector x. ##EQU6##

The amplitude gain g found as the scalar quantity is encoded and decodedindependently, and the coded data together with the index i of thenormalized output vector y_(i) are outputted as the vector quantizationcoded data 310. At the same time, the normalized output vector y_(i) ismultiplied by the local decoding value g of the amplitude gain g therebythe amplitude reconstructed output vector y_(i) * (311) is obtained.

    y.sub.i *=g·y.sub.i

The average data decoded value 309 and the amplitude reconstructedoutput vector 311 are added in the first adder 312 and thereby theresidual signal decoded vector 315 is obtained, and then the residualsignal decoded vector 315 is added in the second adder 316 to theprevious stage vector 319 delayed by one frame. The decoded vector 317of each stage being output of the second adder 316 is written in theframe memory 318 and delayed by one frame. On the other hand, theaverage value coded data 308 and the vector quantization coded data 310are converted into suitable code words respectively in the codeassignment circuit 320 and then transmitted as the coded output data324. At the second stage and so forth, the above-mentioned codingprocesses are repeated and executed one after another with respect tothe residual signal vectors 303 between the input signal vectors 301 andthe previous stage decoded vectors 319 stored in the frame memory 318.

Since the image signal progressive reconstruction coding device isconstituted as above described, the block in coding at each stage isalways the same in size, and when the block size is made larger theoperation scale of the inner product vector quantizer becomes larger inproportion to the dimension number of the vector, and when the blocksize is made smaller it is difficult to decrease the amount of the codedinformation at the first stage significantly.

FIG. 28 is a block diagram showing a composition example of aninterframe vector quantizer in the prior art disclosed, for example, inMurakami et al. "The Vector Quantization Method Interframe CodingSimulation" in the draft 1175 of the annual meeting 1983 of theInstitute of Electronics and Communication Engineers of Japan. In FIG.28, numeral 401 designates the input image signal system, numeral 402designates a subtractor which performs subtraction to the interframepredicting signals, numeral 403 designates the interframe predictingsignals, numeral 404 designates the interframe differential signals,numeral 439 designates a vector quantization coding section, numeral 440designates the coded data, numeral 441 designates a vector quantizationdecoding section, numeral 416 designates decoded interframe differentialsignals, numeral 417 designates an adder which adds the decodedinterframe differential signals 416 and the interframe predictingsignals 403, numeral 418 designates the decoded image signal series,numeral 419 designates a frame memory which gives frame delay to thedecoded image signal series 418 and forms the interframe predictingsignals 403, numeral 420 designates a variable length coding section,numeral 421 designates a buffer for smoothing the speed, numeral 422designates a value, numeral 423 designates a line interface (I/F), andnumeral 424 designates transmission signals.

FIG. 29 is a block diagram showing a composition example of the vectorquantization coding section 439. In FIG. 29, numeral 428 designates anaverage value separation and normalization section, numeral 429designates the normalized vectors, numeral 431 designates a code bookwhich stores the output vectors, numeral 432 designates the outputvectors, numeral 433 designates a distortion operation section whichfinds distortion between the normalized vectors 429 and the outputvectors 432, numeral 436 designates the distortion found in thedistortion operation section 433, numeral 437 designates a minimumdistortion detecting section which detects the minimum value from thedistortion 436, numeral 430 designates the average value and theamplitude which are separated by the average value separation andnormalization section 428, numeral 434 designates a block discriminationsection, numeral 422 designates the threshold value used fordiscriminating the blocks, numeral 435 designates the blockdiscrimination information, numeral 438 designates the index of theoutput vectors giving the minimum distortion, and numeral 440 designatesthe coded data.

The operation of the device will be described. The interframe predictingsignals 403 are subtracted from the input image signal series 401 by thesubtractor 402; thereby the input image signal system 401 is convertedinto the interframe differential signals 404. Since the interframedifferential signal has little power in comparison to the originalsignal, it can be encoded with little coding error. The interframedifferential signal 404 is encoded in the vector quantization codingsection 439 (the coding method is described afterwards). Then thethreshold value 422 is used as a parameter. The coded data 440 encodedin the vector quantization coding section 439 is decoded in the vectorquantization decoding section 441; thereby the decoding predictingdifferential signal 416 is obtained. The interframe predicting signal403 and the decoded interframe differential signal 416 are added in theadder thereby the decoded image signal series 418 is obtained. Thedecoded image signal system 418 is stored temporarily in the framememory 419 and supplied with the frame delay, thereby the interframepredicting signal is formed. On the other hand, the coded data 440 issubjected to variable length coding in the variable length codingsection 420 and stored temporarily in the buffer 421 and subjected tospeed smoothing process, and then passes through the line interface 423and is outputted as the transmission signal 424. In the buffer 421, thethreshold value 422 in proportion to the data storage amount subjectedto variable length coding is outputted and given to the vectorquantization coding section 439 so as to control the information amount.Control of the coding and the information amount in the vectorquantization coding section 439 will be described. The input signals tobe subjected to the vector quantization are the interframe differentialsignals 404. The signals 404 are blocked (vector) in the average valueseparation and normalization section 428, and subjected to the averagevalue separating and normalizing process. If the blocked input signalsare represented as S=[S₁, S₂ , . . . , S_(k) ], the average valueseparating and normalizing process is expressed for example as follows:##EQU7##

The normalized vectors X=[x₁, x₂, . . . , x_(k) ]obtained as abovedescribed are separated from the scalar quantity being the average valueand the amplitude, and therefore unified with respect to the probabilitydistribution in comparison to the vectors S before the average valueseparation and normalization, resulting in the effect to improve theefficiency of the vector quantization as hereinafter described. Thedistortion between the normalized vector 429 and the output vector 432read from the code book 431 is found in the distortion operation section433. In the minimum distortion detecting section 437, the minimum valueamong the distortion 436 between the output vectors stored in the codebook 431 and the input vectors 429 is detected, and the index number 438of the output vector giving the minimum distortion is outputted. Thisprocess is the vector quantization. This is expressed in formulas asfollows: ##EQU8## The maximum inner product value P_(max) is given asthe correction amplitude 430 (hereinafter referred to as "g") whichapproximates x, namely the magnitude of the average value separatedinput vector x as shown in the following formulas. ##EQU9## Thecorrection amplitude g is subjected to the high efficient coding in theamplitude coding device 415, and converted into the amplitude codingdata 418. In the coded data multiplier 420, the average value codingdata 417, the amplitude coding data 418 and the index 419 aremultiplied, and transmitted as the output data 421 for the coding devicein accordance with the prescribed format.

The decoding operation will be described. The output data 421 for thecoding device are separated in the coding data multiplication separatingsection 422 into the average value coding data 417, the amplitude codingdata 418 and the index 419 in accordance with the prescribed format. Theaverage value coded data 417 are decoded through the average valuedecoding device 423, and converted into the average value coded value425 (hereinafter referred to as "μ"). Similarly Y=[y₁, y₂, . . . ,y_(i), . . . , t_(N) ] represents the contents of the code book.##EQU10##

In this case, the coding process is the mapping from x to i, and themapping from i to y_(i) (reading-out of the code book) becomes thedecoding process. i corresponds to the index 438. The average value andthe amplitude 430 are used together with the threshold value 422 todiscriminate the blocks in the block discrimination section 434. Whenthe threshold value 422 is made Th, the block discrimination isexpressed as follows: ##EQU11## As for the insignificant block, theinterframe differential signal of the block is treated as 0.Consequently, the average value, the amplitude 430 and the index 438need not be transmitted in this case. The coding data 440 outputted fromthe vector quantization coding section 439 comprises the average value,the amplitude 430, the block discrimination information 435 and theindex 438, but since the block discrimination information 435 only isvalid in the case of the insignificant block, the information generatingamount can be controlled by the threshold value 422.

Since the interframe vector quantizer in the prior art is constituted asabove described, the control range of the information amount is small.Consequently, if the information amount is suppressed to the minimum,the changed parts on the screen are left behind as the insignificantblocks, resulting in producing so-called "pin hole noise".

FIGS. 33 and 34 are block diagrams showing a composition example of avector quantization coding device and a decoding device using vectorquantization technology in the prior art disclosed, for example, in"Image High Efficiency Coding by the Vector Quantization" in theTechnical Report of the Institute of Electronics and CommunicationEngineers of Japan IT85-61 (1985). In FIG. 33, numeral 501 designatesthe input signal vectors, numeral 502 designates an average valueseparating circuit which separates the average value component withinthe vector from the input signal vectors, numeral 503 designates theaverage value within the vector, numeral 504 designates an average valuecoding section, numeral 505 designates the average value separated inputvectors, numeral 506 designates an inner product operation section whichfinds the inner product value of the average value separated inputvector and the normalized output vectors as hereinafter described,numeral 507 designates a code book ROM which stores a plurality ofnormalized output vectors, numeral 508 designates the address signals,numeral 509 designates the normalized output vectors, numeral 510designates an address counter, numeral 511 designates the inner productvalue calculated in the inner product operation section 506, numeral 512designates a maximum inner product detecting section which finds themaximum value among the plurality of inner product values, numeral 513designates the correction amplitude defined by the maximum inner productvalue, numeral 514 designates the strobe signals, numeral 515 designatesan amplitude coding device, numeral 516 designates an index latch whichtakes the address signals, numeral 517 designates the average valuecoded data, numeral 518 designates the amplitude coded data, numeral 519designates the index, numeral 520 designates a coding data multiplexingsection, and numeral 521 designates the coded output data.

In the composition example of the vector quantization decoding device inFIG. 34, numeral 522 designates a coded data demultiplexing section,numeral 523 designates an average value decoding device numeral 524designates an amplitude decoding device, numeral 528 designates a codebook ROM, numeral 525 designates the average value decoded value,numeral 526 designates the amplitude decoded value, and numeral 527designates the decoded vector.

The operation of the vector quantization coding will be described. Theinput vectors 501 (hereinafter referred to as "S") constituted by theinput signal system blocked every k pieces (k indicates integer being 2or more) are processed in the average value separating circuit 502according to the formulas shown downwards, and subjected to separationof the average value 503 (hereinafter referred to as "μ") within thevectors therefrom and transformed into the average value separated inputvectors 505 (hereinafter referred to as "X").

The input vector S, the average value μ within the vector, and theaverage value separated input vector X are expressed as follows:##EQU12##

The average value μ is subjected to the high efficiency coding in theaverage value coding section 504, and converted into the average valuecoded data 517. The average value separated input vector X is inputtedin the inner product operation section 506, and converted through theinner product vector quantization coding process as hereinafterdescribed into the correction amplitude 513 as hereinafter described andthe index 519 as hereinafter described. The average value separatedinput vector X is normalized with its magnitude ##EQU13## into aplurality of normalized input vectors X*, and N pieces (N being naturalnumber) of normalized output vectors 509 [hereinafter referred to asy_(i) (i=1, 2, . . . , N)] formed according to the statistical propertyof the normalized input vectors X* are writtin in the code book ROM 507.When the average value separated input vector X is inputted in the innerproduct operation section 506, the address counter 510 is reset andstarts the counter operation of the period N. Then the normalized outputvectors y_(i) on the address, which are indicated by the address signals508 outputted from the address counter 510, are inputted sequentiallyfrom the code book ROM 507 into the inner product operation section 506,and the inner product 511 (hereinafter referred to as P(x, y_(i)), i=1,2, . . . , N) between the average value separated input vector X and Npieces of the normalized output vectors y_(i) is calculated according tothe following formula, and then outputted. ##EQU14## Among N pieces ofthe inner product P(x, y_(i)) obtained by the calculation, the maximuminner product value P_(max) is detected in the maximum inner productdetecting section 512, and then the address signals 508 showing theaddress within the code book ROM 507 of the normalized output vectory_(i) giving the maximum inner product value are taken in the indexlatch 516 at timing synchronized with the strobe signals 514. Thetaken-in address signals 508 are the index 519 to discriminate theprescribed normalized output vectors y_(i), and transmitted to thecoding data multiplexing section 520. On the other hand, the maximuminner product value P_(max) is given as correction amplitude 513(hereinafter referred to as "g") to simulate amount |x| of the averagevalue separated input vector x as shown in the following formulas: Thecorrection amplitude g is subjected to the high efficiency coding by theamplitude coding device 515 and converted into amplitude coded data 518.In the coding data multiplexing section 520, the average value codeddata 517, the amplitude coded data 518 and the index 519 are multiplied,and then transmitted as the coding device output data 521 according tothe prescribed format.

The decoding operation will be described. The coding device output data521 are separated in the coded data demultiplexing section 522 accordingto the prescribed format into the average value coded data 517, theamplitude coded data 518 and the index 519. The average value coded data517 are decoded through the average value decoding device 523 andconverted into the average value decoded value 525 (hereinafter referredto as "μ"). In similar manner, the amplitude decoded value 526(hereinafter referred to as "g") are outputted from the amplitudedecoding device 524. From the code book ROM 517 the normalized outputvectors y_(i) being on the address instructed by the index 519 are readout, and the recording vectors 527 (hereinafter referred to as "s") tothe input vector s are obtained through the process of the followingformulas.

    s=[s.sub.1, s.sub.2, . . . , s.sub.k ]

    s.sub.j =g·y.sub.ij +μ

Since the vector quantizer in the prior art is constituted as abovedescribed, it is impossible that the content of the code book is renewedduring coding operation, thereby approximation of the decoded vector maybe deteriorated to the unusual input vectors with different properties.

FIG. 36 shows an example of an image coding method in the prior art. InFIG. 36, the side A indicates the transmitting side and the side Bindicates the receiving side.

In FIG. 36, numeral 601 designates an input buffer which inputs thedigitized image signals and outputs them suitably to the coding sectionat the next stage, numeral 603 designates a frame memory which storesthe image signals after coding and decoding before the present imagesignals by one frame, numeral 602 designates a subtractor which carriesout the subtraction between the output of the input buffer 601 and theoutput of the frame memory 603, numeral 604 designates a quantizationcoding device which gives the quantization and coding to the output ofthe frame memory 603, numeral 605 designates a quantization decodingdevice which decodes the signals after the quantization coding, numeral606 designates an interframe adder which adds the quantization decodingoutput and the output of the frame memory 603, and writes the result tothe frame memory 603, numeral 607 designates a variable length codingdevice which assigns the variable length code to the quantization codingoutput corresponding to the generating frequency of each code, numeral608 designates a transmitting buffer which stores the variable lengthcoded output, numeral 609 designates a transmitting buffer controlsection which monitors the control of writing and reading in thetransmitting buffer 608 and the storage amount of the transmittingbuffer and then transmits the monitoring result to the input buffer 601,numeral 610 designates a dummy data adding section which adds the dummydata to the output of the transmitting buffer, and numeral 611designates a line interface section.

Numeral 612 designates a line interface section on the receiving side,numeral 613 designates a dummy separating section which deletes theadded dummy data, numeral 614 designates a variable length decodingsection which decodes the variable length code, numeral 615 designates areceiving buffer which stores the signals after the variable lengthdecoding, numeral 605' is a quantization decoding section which givesthe quantization decoding to the output of the receiving buffer 615,numeral 603' designates a frame memory which stores the decoded imagesignals before the present image signals by one frame, and numeral 606'designates an interframe adder which adds the output of the quantizationdecoding section 605' and the output of the frame memory 603' and thenwrites the result to the frame memory 603'.

The operation of the device will be described.

The inputted image signals 701 are written to the input buffer 601. Theinput buffer performs writing and reading by the unit of the imageframe, but it has the composition of double buffer because reading maybe performed during writing.

The coded and decoded image signals 702 before the present image signalsby one frame are outputted from the frame memory 603. In the interframesubtractor 602, the interframe differential signals 703 are obtained bysubtracting between the present image signals 701' read from the inputbuffer and the image signals 702. The interframe differential signals703 are encoded by the quantization coding device 604, and become thequantization coded signals 704. FIG. 37 shows an example ofcharacteristics of the quantization coding device. The quantizationcoded signals 704 are inputted in the variable length coding device 607,and transformed into the variable length code 706 corresponding to thefrequency of each coded signal.

At the same time, the quantization coded signals 704 are inputted in thequantization decoding device 605, and then are outputted as the codedand decoded differential signals 705. FIG. 38 shows an example ofcharacteristics of the quantization decoding device.

The coded and decoded differential signals 705 are inputted togetherwith the image signals 702 into the interframe adder 606, and become thecoded and decoded image signals 702 and are written to the frame memory603 for the coding to the next frame.

On the other hand, the variable length codes 706 are inputted in thetransmitting buffer 608. The transmitting buffer outputs the data inaccordance with the requirement from the transmission line side afterstoring the variable length codes over the definite amount, and has thecomposition of double buffer (buffer #1, buffer #2) because writing andreading must be performed at the same time. The transmitting buffercontrol section 609 controls writing and reading of the transmittingbuffer. For example, when the buffer #1 is at writing operation and thebuffer #2 is at reading operation, the transmitting buffer controlsection 609 monitors the storage amount of the buffer #1, and if thestorage amount becomes more than the prescribed set value, thetransmitting buffer control section 609 demands ceasing of output of thedata to the input buffer 601.

Receiving the demands, the input buffer 601 ceases the output of thedata to the next stage. The transmitting buffer control section 609detects the pause of the input data to the transmitting buffer 608, andceases writing to the buffer #1 and makes the situation of waiting forreading. The buffer #2 during reading ceases reading if the residualamount becomes less than the prescribed set value, and it waits for thebuffer #1 to be in the situation of waiting for reading. When the buffer#1 is in the situation of waiting for reading, the buffer #2 and thebuffer #1 are read out continuously. The buffer #2 is in the situationof waiting for writing when the residual amount becomes zero.

When the buffer #2 is in the situation of waiting for writing, thetransmitting buffer control section 609 demands to start the output ofthe data to the input buffer 601.

In this process, before the buffer #1 gets in the situation of waitingfor reading, there becomes the situation that the transmitting buffer608 cannot output any data.

The dummy data adding section 610 outputs the data with the dummy dataadded thereto so as to continue the transmission of the data to thetransmission line without break while the transmitting buffer 608 cannotoutput the data.

The data with the dummy data added thereto are subjected to conversionof electric level in the transmission line interface section 611 so asto meet the characteristics of the transmission line, and then outputtedto the transmission line. On the receiving side, the signals inputtedthrough the transmission line are subjected to phase conversion ofelectric level in the transmission line interface section 612, and thedummy data added in the dummy data adding section 610 are cleared in thedummy separating section 613; thereby only the data about the images areoutputted.

The output is processed in the variable length decoding section 614 bythe reverse treatment with respect to that in the variable length codingdevice 607, and then inputted in the form of the quantization codedsignals 704 into the receiving buffer 615.

The receiving buffer has the composition of double buffer, becausewriting and reading are performed at the same time. In the receivingbuffer, the stored data are variable in amount so as to take matchingwith respect to time between the signal speed inputted from thetransmission line side and the speed of the image decoding section atthe rear stage.

For example, if the processing speed of the image decoding section atthe next stage is low, the stored amount of the receiving buffer isincreased. On the contrary, if the processing speed is high, thereceiving buffer acts at the small stored amount.

The quantization coded signals 704 are decoded by the quantizationdecoding device 605' and outputted as the coded and decoded differentialsignals 705' in similar manner to the transmitting side.

The coded and decoded image signals 702' before the present decodingimage by one frame are outputted from the frame memory 603', and areadded to the coded and decoded differential signals 705' in theinterframe adder 606', and the resulting signals are written as thecoded and decoded image signals to the frame memory 603' and alsooutputted to outside.

Let the necessary time to decode one image memory on the receiving sidebe T_(D), and let the average time to encode one image memory on thetransmitting side be T_(c), and unless T_(D) ≦T_(c), the data areaccumulated one after another in the receiving buffer resulting inoverflowing.

Since the transmitting side transmits the variable length coding data,the number of the data transmitted by one frame is not kept constant.

In other words, the time interval T_(B) at which the code corresponds tothe lead of the image frame can be T_(B) <T_(D) once in a while.

Accordingly, unless T_(D) ≧min T_(B) on the receiving side, it causesthe overflowing of the receiving buffer.

Since the image coding device in the prior art is constituted as abovedescribed, one frame decoding time T_(D) on the receiving side isinevitably set to a small value so that T_(D) <min T_(B). As a result,the scale of the device on the receiving side becomes much larger thanthat on the transmitting side.

SUMMARY OF THE INVENTION

In order to eliminate the above-mentioned disadvantages in the priorart, a first object of the invention is to provide an image coding andtransmitting device which suppresses the predicting error component andthe amount of the generating information, and prevents the deteriorationof the picture quality in case of the great movement of the image oreven in case of scene change.

A second object of the invention is to provide an image signalprogressive build-up coding and decoding device which controls suitablythe amount of the coded information and the reconstructed image qualityover a wide range.

A third object of the invention is to provide an interframe codingdevice which can give the interframe coding by always using the suitablethreshold value without delaying in controlling the threshold value.

A fourth object of the invention is to provide an image signalprogressive build-up coding device which can vary the block size withoutincrease of the operation scale, and control the amount of the codedinformation at every stage over a wide range.

A fifth object of the invention is to provide an interframe vectorquantizer which can widely control the amount of the information withoutdeteriorating the quality of the reconstructed image too much.

A sixth object of the invention is to provide a vector quantizer whichcan produce the decoded vector with high approximation even against theunusual input vector.

A seventh object of the invention is to provide an image coding methodwhich can communicate normally without the buffer overflowing even incase that the device scale on the receiving side is small while T_(D) iscomparatively large.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating inner composition of a motioncompensation circuit in a first embodiment of the invention;

FIG. 2 is a diagram illustrating steps of process of the motioncompensation circuit in FIG. 1;

FIG. 3 is a block diagram of a transmitting section of an image codingtransmitting section in the prior art;

FIG. 4 is a block diagram illustrating inner composition of the motioncompensation circuit in FIG. 3;

FIG. 5 is an arrangement diagram of searching vectors in the motioncompensation circuit in FIG. 3;

FIG. 6 is a block diagram of a coding section of an image signalprogressive build-up coding and decoding device in a second embodimentof the invention;

FIG. 7 is a block diagram illustrating composition of the decodingsection in FIG. 6;

FIG. 8 is a diagram illustrating inner product vector quantization;

FIG. 9 is a diagram illustrating transmitting sequence of coded outputdata in this embodiment;

FIG. 10 is a block diagram of a coding section of an image signalprogressive build-up coding and decoding device in the prior art;

FIG. 11 is a block diagram illustrating composition of a decodingsection in FIG. 10;

FIG. 12 is a block diagram illustrating composition of a transmittingsection of an interframe coding device in a third embodiment of theinvention;

FIG. 13 is a block diagram illustrating composition of the temporalfilter in FIG. 12;

FIG. 14 is a block diagram illustrating composition of the codingcontrol section in FIG. 12;

FIG. 15 is a diagram illustrating operation of coding control in thisembodiment;

FIG. 16 is a block diagram illustrating composition of a transmittingsection of an interframe coding device in the prior art;

FIG. 17 is a block diagram illustrating composition of the codingcontrol section in FIG. 16;

FIG. 18 is a diagram illustrating operation of coding control in theinterframe coding device in FIG. 16;

FIG. 19 is a block diagram illustrating composition of a transmittingsection of an image signal progressive build-up coding device in afourth embodiment of the invention;

FIG. 20 is a diagram illustrating operation of the vector/subvectorconverter in FIG. 19;

FIG. 21 is a diagram illustrating operation of the subvector/vectorconverter in FIG. 19;

FIG. 22 is a diagram illustrating an example of control means of thecoding control section in FIG. 19;

FIG. 23 is a block diagram illustrating composition of a transmittingsection of an image signal progressive build-up coding device in theprior art;

FIG. 24 is a diagram illustrating operation principle of an innerproduct vector quantizer;

FIG. 25 is a block diagram illustrating composition of an interframevector quantization coding device in a fifth embodiment of theinvention;

FIG. 26 is a diagram illustrating relation between the first stagevector quantization and the second stage vector quantization in theinterframe vector quantization coding device in FIG. 25;

FIG. 27 is a block diagram illustrating composition of the first stagevector quantization coding section of the interframe vector quantizationcoding device;

FIG. 28 is a block diagram illustrating composition of an interframevector quantization coding device in the prior art;

FIG. 29 is a block diagram illustrating composition of the vectorquantization coding section in FIG. 28;

FIG. 30 is a block diagram illustrating composition of a coding sectionof a dynamic vector quantizer in a sixth embodiment of the invention;

FIG. 31 is a block diagram illustrating composition of a decodingsection of the dynamic vector quantizer in FIG. 30;

FIG. 32 is a block diagram illustrating composition of the dynamic codebook in FIG. 30;

FIG. 33 is a block diagram illustrating composition of coding section ofa vector quantizer in the prior art;

FIG. 34 is a block diagram illustrating composition of a decodingsection in FIG. 33;

FIG. 35 is a block diagram of image coding transmission system accordingto the invention;

FIG. 36 is a block diagram of image coding transmission system in theprior art;

FIG. 37 is a diagram illustrating an example of characteristics of thequantization coding device;

FIG. 38 is a diagram illustrating an example of characteristics of thequantization decoding device; and

FIG. 39 is a diagram illustrating operation state of the transmittingbuffer.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A first embodiment of the invention will be described referring to FIG.1-FIG. 2.

In FIG. 1, reference numeral 20 designates a quantizer which quantizesthe average value of the input block signals, numeral 23 designates ablock selector which changes and outputs a block where previous frameblock signals and picture elements within the block are 0 (hereinafterreferred to as "0 block") and an average value block, and numeral 25designates an index selector which changes and outputs motion vectorindex and quantized level index. Numerals 1-18 are similar to the priorart.

The operation of the device will be described.

In FIG. 1, the interblock matching distortion operation and thecomparison operation finish in serial processing with (L+2) times perone input block when the number of the searched vectors is made L (Lbeing plus integer). An example of processing steps is shown in FIG. 2.

Let the input block signals 1 be S={S₁, S₂, . . . , S_(K) }, and let thereference block signals 24 be S'_(i) ={S'_(i1), S'_(i2), . . . , S'_(iK)}, and the calculation algorithm of the matching distortion d_(i) issimilar to the prior art.

In the first processing, the block selector 23 outputs "0" block. Inthis case, an output 17 of a distortion operator 16 is represented byfollowing formula, and this value becomes the average value m of theinput signal blocks. However, the matching distortion operation uses theluminance signals only. ##EQU15##

The average value m is quantized in the quantizer 20, and thequantization average value m 22 is inputted in the block selector 23. Onthe other hand, the quantization index 21 indicating the quantized levelof the quantized average value is outputted to the index selector 25.Let the number of the quantized level be M (M being plus integer), andthe quantized index from (L+1) to (L+M) is assigned at each level.

In the second processing, the block selector 23 outputs the averagevalue block m having the quantized average value m as value of eachpicture element within the block. The output 17 of the distortionoperator 16 in this case is represented by following formula, and thisvalue is made d_(o). ##EQU16## where 0 is given as the vector index ofthe average value block.

In the third processing and so forth, the distortion operation and thecomparison operation are the same as noted above. That is, the blockselector 23 outputs the previous frame block one after another.

Finally, the output 19 of the comparator 18 becomes the index of theblock which gives the minimum matching distortion among (L+1) piecesfrom d_(o) to d_(L) of the matching distortion. In this case, the vectorindex corresponding to each reference block is assigned as shown in FIG.2.

In the index selector 25, when the comparator output index 19 is 0,i.e., when the average value block m is selected, the quantized indexwhich newly indicates the quantized level of the quantized average valuem [the value being from (L+1) to (L+M)] is outputted as the vector index5. In other cases, the value of the comparator output index 19, as itis, is outputted as the vector index 5.

In this embodiment, although the differential absolute sum isrepresented as the interblock matching distortion, Euclid norm may beused in place of it, and it is possible to increase the priority of thespecific reference block by giving weight to the matching distortion ineach reference block.

A second embodiment of the invention will be described referring to FIG.6-FIG. 11.

In FIG. 6, numeral 101 designates input signal vectors which block theinput image signal series at each sample m×n (m, n being integers),numeral 102 designates a subtractor which estimates residual signalvector between the input signal vectors and previous stage decodingvectors delayed by one frame, numeral 103 designates the residual signalvectors, numeral 104 designates a switch which controls ON/OFF operationof a sample value coding and decoding device, an average value codingand decoding device and a normalized vector quantization coding anddecoding device according to control signals from a coding controlsection, numeral 105 designates a sample value coding and decodingdevice which gives coding and decoding to sample value in the prescribedcoding mode controlled according to control signals from the codingcontrol section per each sample, numeral 106 designates an average valuecoding and decoding device which finds the average value within theblock per each vector, and gives coding and decoding to the averagevalue in the prescribed coding mode controlled according to controlsignals from the coding control section, numeral 107 designates anormalized vector quantization coding and decoding device which givesthe inner product vector quantization in the prescribed coding modecontrolled according to control signals from the coding control per eachvector, numeral 108 designates coded data of the sample value, numeral109 designates decoded value of the sample value, numeral 110 designatescoded data of the average value, numeral 111 designates decoded value ofthe average value, numeral 112 designates coded data of the amplitudeand the output vector index, numeral 113 designates output vectorssubjected to amplitude reconstruction, numeral 114 designates a codeassignment circuit which assigns codes to each coded data according tocontrol signals from the coding control section, numeral 115 designatesoutput signals of the coding device, numeral 116 designates a vectorreconstruction circuit which obtains residual signal decoded vectors inthe prescribed operation according to control signals from the codingcontrol section, numeral 117 designates residual signal decoded vectors,numeral 118 designates an adder which adds the residual signal decodedvectors and the previous stage decoded vectors delayed by one frame,numeral 119 designates decoded vectors, numeral 120 designates a framememory which delays the decoded vector by one frame, numeral 121designates a coding control section which generates control signals tocontrol each coding mode of the switch, the sample value coding anddecoding device, the average value coding and decoding device and thenormalized vector quantization coding device corresponding to errorbetween the input signal vector and the decoded vector and the number ofcoding stages, numeral 122 designates control signals from the codingcontrol section, and numeral 123 designates coding stage indicatingsignals representing the number of coding with the frame unit.

In FIG. 7, numeral 124 designates a code assignment circuit whichcarries out the code assignment inversion according to control signalsfrom the coding control section, numeral 125 designates a sample valuedecoding device which carries out the sample value decoding by theprescribed decoding mode controlled according to control signals fromthe coding control section per each sample, numeral 126 designates anaverage value decoding device which carries out the average valuedecoding by the prescribed decoding mode controlled according to controlsignals from the coding control section, and numeral 127 designates anormalized vector quantization decoding device which carries out thevector quantization decoding by the prescribed decoding mode controlledaccording to control signals from the coding control section.

The operation of the device will be described.

In FIG. 6, in the previous stage coding, the frame memory 120 storingthe previous stage decoding results remains cleared, and the inputsignal vectors 101, as they are, pass through the subtractor 102 and areoutputted as the residual signal vectors 103. The switch 104 rendersonly the average value coding and decoding device 106 to ON-stateaccording to the control signals 122 from the coding control section. Inthe average value coding and decoding device 106, the average valuewithin the block is calculated per each vector, and subjected to codingand decoding in the coding mode controlled according to the controlsignals 122, and the coded data 110 of the average value and the decodedvalue 111 are outputted. The coded data 110 of the average value aretransformed by the code assignment circuit 114 into suitable code words,and then transmitted together with the control signals 122 in sequenceof the first stage transmission data shown in FIG. 9. The vectorreconstruction circuit 116 outputs the residual signal decoded vectors117 having all m×n samples as the decoded value 111 of the average valueaccording to the control signals 122 from the coding control section.The residual signal decoded vectors 117 pass through the adder 118, andare written as the decoded vectors 119 into the frame memory 120. At thesecond stage coding, in the subtractor 102, the residual signal vectors103 by subtraction of the previous stage decoded vectors from the inputsignal vectors 101 are obtained. The switch 104 renders the averagevalue coding and decoding device 106 and the normalized vectorquantization coding and decoding device 107 to ON-state according to thecontrol signals 122 from the coding control section. In the averagevalue coding and decoding device 106, the average value within the blockis calculated per each vector, and the coded data 110 of the averagevalue and the decoded value 111 are outputted in the coding modecontrolled according to the control signals 122. In the normalizedvector quantization coding and decoding device 107, the inner productvector quantization is performed, and the coded data 112 of theamplitude and the output vector index and the output vector 113regenerated in terms of the amplitude are outputted.

The operation principle of the inner product vector quantization will bedescribed referring to FIG. 8.

In the inner product vector quantization, the average value separationand normalization is given, and a set of the normalized output vectory_(i) arranged on the unit superspherical surface of themultidimensional signal space, Y=[y₁, y₂, . . . , y_(N) ] is used. Thatis, the normalized output vector y_(i) satisfies the followingconditions simultaneously: ##EQU17##

The input signal vector x is converted through the operation process ofthe following formulas into the normalized output vector y_(i) whichgives the maximum inner product value to the input signal vector x, andthe maximum inner product value is given as the amplitude g of the inputsignal vector x. ##EQU18##

The amplitude g found as the scalar quantity is encoded independently,and the coded result together with the index i of the normalized outputvector y_(i) are outputted as the coded data. At the same time, thenormalized output vector y_(i) is multiplied by the decoded value g ofthe amplitude g; thereby the amplitude reconstructed output vectory_(i) * is obtained.

    y.sub.i *=g·y.sub.i

The coded data 110, 112 are transformed by the code assignment circuit114 into suitable code words, and are transmitted together with thecontrol signals 122 in sequence of the second stage transmission datashown in FIG. 9. In the vector reconstruction circuit 116, the averagevalue decoded vectors having all m×n samples as the decoded value 111 ofthe average value and the amplitude reconstructed output vectors 113 areadded according to the control signals 122 from the coding controlsection, and the residual decoded vectors 117 are outputted. Theresidual signal decoded vectors 117 are added in the adder 118 to theprevious stage decoded vectors, and the decoded vectors 119 as theadding result renew the frame memory 120. At the third stage coding, theresidual signal vectors 103 are outputted from the subtractor 102 insimilar manner to the second stage coding. The switch 104 renders onlythe sample value coding and decoding device 105 to ON-state according tothe control signals 122 from the coding control section. In the samplevalue coding and decoding device 105, coding and decoding of the samplevalue are executed per each sample within the vector in the coding modecontrolled according to the control signals 122, and the coded data 108of the sample value and the decoded value 109 are outputted. The codeddata 108 of the sample value are transformed by the code assignmentcircuit 114 into suitable code words, and transmitted together with thecontrol signals 122 in sequence of the third stage transmission datashown in FIG. 9. In the vector reconstruction circuit 116, according tothe control signals 122 from the coding control section, the residualsignal decoded vectors 117 composed of the decoded value 109 m×n piecesof the sample values are outputted. The residual signal decoded vectors117 are added in the adder 118 to the previous stage decoded vectors,and the decoded vectors 119 as adding result renew the frame memory 120.The same coding operation as that of the third stage is repeatedafterwards, and the coded data 108 of the sample value at each stage aretransmitted so as to renew the frame memory 120.

On the other hand, in the decoding section, when the decoding starts,the frame memory 120 storing the previous stage decoded result remainscleared, and the coded output signals 115 obtained in the coding sectionare subjected to the code assignment process in the code assignmentcircuit 124 and obtained as each coded data. Each coding data is decodedby each of the decoders 125, 126, 127 according to the prescribed modeaccording to the control signals 122 transmitted at the same time fromthe coding section, and then outputted as the residual signal decodingvectors 117 from the vector reconstruction circuit 116. The residualsignal decoded vectors 117 are added to the previous stage decodedresults in the adder 118 so as to renew the frame memory 120.

In the coding control section 121, the average of the error between theinput signal vector 101 and the decoded vector 119 in each stage isestimated in one frame period, and in accordance with the average errorand the coded stage indicating signals 123, the control signals 122 forfeedback control of changing of the switch 104 and changing of theaverage value coding mode, the normalized vector quantization codingmode and the sample value coding mode in the next stage coding aregenerated and outputted.

Let the input signal vector be S and the decoded vector be S, then theerror e may be defined, for example, by the following formula: ##EQU19##

In this embodiment, although the average of the error between the inputsignal vector 101 and the decoded vector 119 in one frame period is usedin the coding control section 121 and the feedback control is executed,the error between the residual signal vector 103 and the residual signaldecoded vector 117 may be used, and in place of the average of the errorin one frame period, the amount of all coded information of one frameperiod may be used so as to obtain the same effect as the embodiment.

FIG. 12 is a block diagram illustrating composition of a transmittingsection of an interframe coding device in a third embodiment of theinvention. In FIG. 12, numeral 201 designates a digitalized motion imagesignal system, numeral 202 designates a temporal filter, numeral 203designates interframe differential signals obtained in the temporalfilter 202 and subjected to the non-linear weighting, numeral 204designates a motion amount estimation section, numeral 205 designatesestimated motion amount obtained in the motion amount estimation section204, numeral 206 designates motion image signal system which isprocessed by the temporal filter 202 and subjected to conversion of theresolving power or the speed, numeral 207 designates a subtractor forobtaining interframe differential signals, numeral 208 designatesinterframe predicting signals formed from the past-time frame afterfinishing the coding already, numeral 209 designates interframedifferential signal series, numeral 210 designates a blockdiscrimination section which quantizes the interframe differentialsignal 209 being the predicting error as zero, and discriminates whetherthe coding processing at the next stage must be performed or not blockby block unit, numeral 211 designates threshold value for the blockdiscrimination, numeral 212 designates a coding control section whichdetermines the threshold value, numeral 213 designates a coding anddecoding section which encodes and decodes a block discriminated to havethe significant predicting error in the block discrimination section210, i.e., significant block, numeral 214 designates decoded predictingerror signals obtained in the coding and decoding section 213, numeral215 designates an adder which adds the predicting signals 208 and thedecoded predicting error signals 214, numeral 216 designates decodedimage signals, numeral 217 designates a frame memory which forms theinterframe predicting signals 208 using the decoded image signals 216,numeral 218 designates coded data obtained in the coding and decodingsection 213, numeral 219 designates a variable length coding sectionwhich gives the variable length coding to the coded data 218, numeral220 designates a buffer where the data series subjected to the variablelength coding in the variable length coding section 219 is transmittedat the definite transmission speed, numeral 221 designates data bycounting the variable length coded data per each frame (informationamount per frame), numeral 222 designates a line interface where thedata system smoothed in the buffer 220 is transmitted to thetransmission line, and numeral 223 designates transmitting signals. FIG.13 is a block diagram illustrating composition of the temporal filter202 in FIG. 12.

In FIG. 13, numeral 224 designates a subtractor which finds differencebetween frames, numeral 225 designates image signals of the past-timeframe, numeral 226 designates interframe differential signals, numeral227 designates a non-linear weighting circuit which provides weight tothe interframe differential signals 226, numeral 228 designates an adderwhich adds the interframe differential signals 203 weighted and theimage signals 225 of the past-time frame, numeral 229 designatestemporal filter output signals obtained by the adder 228, and numeral230 designates a frame memory which performs frame delay, resolvingpower conversion or speed conversion.

FIG. 14 is a block diagram illustrating composition of the codingcontrol section 212 in FIG. 12. In FIG. 14, numeral 231 designates atable ROM, numeral 232 designates a register which delays the thresholdvalue 211 at the frame period, and numeral 233 designates thresholdvalue delayed by the register 232.

FIG. 15 is a diagram illustrating characteristics to be written in thetable ROM 231.

The operation of the device will be described. The digitalized imagesignal series 201 is first inputted to the temporal filter 202. In thetemporal filter 202, subtraction to the previous frame signal 225 isperformed by the subtractor 224 and the output is converted into theinterframe differential signal 226. The interframe differential signal226 is made the weighted signal 203 by the non-linear weighting circuit227, and added to the previous frame signal 225 by the adder 228 intosignal 229 subjected to the temporal filter processing. The signal 229is stored in the frame memory 230, and used in the filter processing ofnext frame. The frame memory 230 not only gives the frame delay for thetemporal filter, but also has function of converting the resolving poweror the speed to meet the specification of the coding section or thetransmission speed and reading it as the coding input image signals 206.When the transmission speed is low, the coding frames are thinned out,i.e., so-called frame cutting is performed. Then only the frames forcoding are read from the frame memory 230, and other frames removed bythe frame cutting are overwritten on the memory. Consequently, the timerelation between the image signal series 201 and the previous framesignal 225 becomes relation of "frames which are encoded continuously".The weighted interframe differential signal 203 is integrated by oneframe in the motion amount estimation section 204, and used as theestimated motion amount 205 for determining the threshold value ashereinafter described. The signal 206 read from the frame memory 230 issubjected to subtraction to the interframe predicting signal 208 by thesubtractor 207, and converted into the interframe differential signal209. The interframe predicting signal 208 is formed using thereconstruction image signals of the past-time frame after finishing thecoding and the local decoding in the transmitting section already, andmethod of the motion compensation or the like may be used in this case.Since the interframe differential signal 209 gets quite close to 0 whenthe subject has no variation or motion, the significant block and theinsignificant block are discriminated in the block discriminationsection 210 as above described, and regarding the insignificant blockthe data indicating it as insignificant only are transmitted thereby theinformation amount can be compressed. In order to discriminate blocks,the absolute value sum of the interframe differential signal systemformed in blocks is estimated, and then compared with the thresholdvalue 211. Let the interframe differential signal series in blocks beε_(j) (j=1, 2, . . . , k), and the threshold be T_(n), then the blockdiscrimination is performed as follows: ##EQU20##

Regarding the block discriminated as a significant block, further theinterframe differential signal 209 is encoded. In the coding anddecoding section 213, the significant block is encoded and then decoded,and the decoded predicting error signal 214 and the coded data 218 areoutputted. There are various sorts of the coding and decoding methods.However, since these have no direct relation to the essence of theinvention, the detailed description shall be omitted here. The decodingpredicting error signal 214 is added to the interframe predicting signal208 by the adder 215 so that the decoded image signals 216 are formed.The decoded image signals 216 are stored in the frame memory 217, andused to form the interframe predicting signal in the next frame and soforth. On the other hand, the coded data 218 is assigned with code wordscorresponding to the frequency of each data in the variable lengthcoding section 219. In the buffer 220, the speed is smoothed so that thecoded data series subjected to the variable length coding is transmittedat the constant transmission speed, and the coded data series invariable length coding is counted per each frame and outputted as theinformation amount 221 per frame to the coding control section 212. Theline interface 222 transmits the variable length coded data with speedsmoothed as the transmitting signals 223 to the transmission line.

In the coding control section 212, using the estimated motion amount205, the information amount 221 and the threshold value 233 suppliedwith the frame delay by the register 232 and referring to the table ROM234, the new threshold value 211 is transmitted. Let the estimatedmotion amount 205 be M_(n), the past threshold value 233 be T_(n-1), andthe information amount 221 in the frame coded using T_(n-1) be B_(n-1),then the new threshold value T_(n) (211) is obtained as shown in FIG.15.

Between the threshold value and the information amount, there ishyperbolic relation as shown by four curves in FIG. 15. The fourcharacteristic curves show an example, and vary depending on whether themotion of the subject is large or small. First, from B_(n-1) andT_(n-1), one characteristic curve is selected among a plurality ofcurves stored in the table ROM 231. The characteristic curve indicateswhat degree of the motion is included in the frame which is encodedusing the threshold value T_(n-1) and generates the information ofB_(n-1). Since the estimated motion amount M_(n) (205) estimates themotion amount of the frame to be encoded hereafter, the motioncharacteristic curve is suitably transferred using M_(n). If M_(n) islarge, the curve is transferred to the characteristics with largemotion; if M_(n) is small, the curve is transferred to that with smallmotion. Subsequently the point with the information amount being B* issearched along the determined curve. In this case, B* is the informationamount allowed per one frame corresponding to the transmission speed. Ifthe point with the information amount being B* is determined on theselected curve, the threshold value on the point is read as T_(n). Thatis, the characteristic curve corresponding to the motion amount isselected from relation between the threshold value T_(n-1) of the frameafter finishing the coding and the information amount B_(n-1). Regardingthe frame to be coded hereafter, the characteristic curve is correctedusing the estimated motion amount, and control is performed so that thethreshold value required to attain the intended information amount isobtained as the new threshold value T_(n) from the curve.

In this embodiment, although the temporal filter processing or the framedelay by the frame memory 230 to convert the resolving power and thespeed is used as means for obtaining the interframe differential signal203 being the reference signal to estimate the motion amount, of course,the frame memory may be prepared only for estimating the motion amount.

A fourth embodiment of the invention will be described referring to FIG.19-FIG. 22. FIG. 19 is a block diagram illustrating composition of atransmitting section of an image signal progressive build-up codingdevice in the fourth embodiment of the invention. In FIG. 19, numeral304 designates a vector/subvector converter which reduces the dimensionnumber of the input vector from L (=am×bn), (a, b, m, n being naturalnumber) into K (=m×n) and forms subvector, numeral 314 designates asubvector/vector converter which uses a sample value within thesubvector of the dimension number K (=m×n) and forms the vector of thedimension number L (=am×bn) in interpolation reconstruction, numeral 305designates residual signal subvector, numeral 313 designates residualsignal decoding subvector, numeral 322 designates a coding controlsection which produces control signals 323 indicating parameters a, b,m, n to determine the dimension number L and K of the vectors and thethreshold value corresponding to coded stage number 321, and numeral 325designates discrimination information which indicates whether the codingand decoding to the residual signal vector 303 should be executed ornot. Other numerals 301-303, 306-312, 314-320, 324 are similar to thosein the device of the prior art.

The operation of the device will be described. In FIG. 19. at the firststage coding, the frame memory 318 remains cleared. The input signalvector 301 of the dimension number L =(am×bn) is not subjected to anyprocessing in the subtractor 302, but outputted as it is as the residualsignal vector 303. The residual signal vector 303 is inputted to thevector/subvector converter 304, and subjected to the conversionprocessing of the dimension number as hereinafter described andconverted into the residual signal subvector 305 of the dimension K(=m×n) and then outputted. In the vector/subvector converter 304, theresidual signal vector 303 and the residual signal subvector 305 arerepresented respectively as follows: ##EQU21## The sample value u_(j) inthe subvector is given by value of samples in the input vector averagedper a×b samples as shown in the following formulas: ##EQU22##

FIG. 20 shows relation between the input vector and the subvector whena=b=m=n=4 corresponding to the image signal series of two dimensions.

The residual signal subvector 305 is inputted to the average valuecoding device 306 and the normalized vector quantization coding device307, and subjected to the same processing as that of the prior art. As aresult, the average value coded data 308 and the average value decodedvalue 309 are outputted from the average value coding device 306, andthe vector quantization coded data 310 and the amplitude constructedoutput vector 311 are outputted from the normalized vector quantizationcoding and decoding device 307, and the average value decoded value 309and the amplitude reconstructed output vector 311 are added in the firstadder 312; thereby the residual signal decoded subvector 313 isobtained. The residual signal decoded subvector 313 of K dimensions issubjected to the following processing through the subvector/vectorconverter 314, and converted into the residual signal decoded vector 315of L dimensions. In the subvector/vector converter 314, the residualsignal decoded subvector 313 and the residual signal decoded vector 315are represented respectively as follows: ##EQU23## The sample s_(r) inthe output vector is given in that the same samples in the subvector arearranged repeatedly per a×b samples as shown in the following formulas:##EQU24##

FIG. 21 shows the sample arrangement in two dimensions of the subvectorcorresponding to samples in the output vector when a=b=m=n=4.

The residual signal decoded vector 315 is added in the second adder 316to the previous stage decoding vector 319 delayed by one frame insimilar manner to the device in the prior art, and written as thedecoded vector 317 in the frame memory 318 and used for the coding inthe next stage. The average value coded data 308 and the vectorquantization coded data 310 are supplied to the code assignment circuit320.

At the second stage and so forth, to the residual signal vector 303between the input signal vector 301 and the previous stage decodedvector 319 stored in the frame memory 318, the above-mentioned codingprocessing is executed in sequence repeatedly under following control:

The coding control section 322 produces the parameters a, b, m, n todetermine the dimension number of each of the residual signal vector303, the residual signal subvector 305, the residual signal decodedsubvector 313, the residual signal decoded vector 315 in the codingprocessing and the control signal 323 indicating the threshold value ashereinafter described corresponding to the coded stage number 321, andsupplies the control signals 323 to the vector/subvector converter 304,the subvector/vector converter 314 and the code assignment circuit 320.FIG. 22 shows an example of setting method of the dimension number L(=am×bn) of the residual signal vector 303 at the first stage and thesecond stage corresponding to the image signal system of two dimensions.At the second stage coding processing and so forth, the vector/subvectorconverter 304 compares the average square value per sample within theresidual signal vector 303 with the threshold value supplied from thecoding control section 322 in magnitude. As a result of comparison, ifthe average square value is larger than the threshold value, the codingand decoding processing is executed. If not, the coding and decodingprocessing is not performed, but all sample values within the residualsignal decoded vector 315 outputted from the subvector/vector converter314 are made 0. The discrimination information 325 indicating result ofthe magnitude comparison is supplied from the vector/subvector converter304 to the code assignment circuit 320 and the subvector/vectorconverter 314.

In the code assignment circuit 320, the control signal 323, thediscrimination information 325, the average value coded data 308 and thevector quantization coded data 310 are converted into suitable codewords, and then transmitted as the coding output data 324.

In this embodiment, although a plurality of samples of the input vectorare combined and averaged and made samples of the subvector in theoperation of the vector/subvector converter, samples within the inputvector may be extracted according to suitable subsample pattern and madesamples of the subvector, and further a smoothing filter for the bandlimiting may be inserted in the input stage.

A fifth embodiment of the invention will be described referring to FIG.24-FIG. 28. FIG. 25 is a block diagram illustrating composition of acoding section of an interframe vector quantization device. In FIG. 25,numeral 401 designates input image signal series, numeral 402 designatesa subtractor which performs subtraction to interframe predictingsignals, numeral 403 designates interframe predicting signals, numeral404 designates interframe differential signals, numeral 405 designates afirst stage vector quantization coded section, numeral 406 designatesfirst stage coding data, numeral 407 designates a first stage vectorquantization decoding section, numeral 408 designates first stagedecoded signals, numeral 409 designates a subtractor which findsdifference between the interframe differential signals 404 and the firststage decoded signals 408, numeral 410 designates first stage errorsignals, numeral 411 designates a second stage vector quantizationcoding section, numeral 412 designates second stage coded data, numeral413 designates a second stage vector quantization decoding section,numeral 414 designates second stage decoded signals, numeral 415designates an adder which adds the first stage decoded signals 408 andthe second stage decoded signals 414, numeral 416 designates decodinginterframe differential signals, numeral 417 designates an adder whichadds the interframe predicting signals 403 and the interframedifferential signals 416, numeral 418 designates decoded image signalseries, numeral 419 designates a frame memory which supplies the decodedimage signal system 18 with the frame delay and forms the interframepredicting signals 403, numeral 420 designates a variable lengthdecoding section, numeral 421 designates a buffer for the speedsmoothing, numeral 422 designates threshold value, numeral 423designates a line interface (IF), and numeral 424 designates transmittedsignals. FIG. 26 is a diagram illustrating relation between the firststage vector quantization and the second stage vector quantization. FIG.27 is a block diagram illustrating composition of the first stage vectorquantization coding section 405 in FIG. 25. In FIG. 27, numeral 425designates an average value and amplitude operation section, numeral 426designates average value, numeral 427 designates average value andamplitude, numeral 428 designates an average value separation andnormalization section, numeral 430 designates average value andamplitude, numeral 429 designates normalized vectors, numeral 431designates a code book which stores output vectors, numeral 432designates output vectors, numeral 433 designates a distortion operationsection which calculates distortion between the normalized vectors 429and the output vectors 432, numeral 422 designates threshold value,numeral 434 designates a block discrimination section, numeral 436designates distortion calculated in the distortion operation section433, numeral 437 designates a minimum distortion detecting section whichdetects the minimum value of the distortion 436, numeral 435 designatesblock discrimination information, and numeral 438 designates index ofthe output vector giving the minimum distortion.

The operation of the device will be described. The input image signalseries 401 is subjected subtraction of the interframe predicting signal403 by the subtractor 402, and converted into the interframedifferential signal 404. The interframe differential signal has littlepower in comparison to the original signal, and therefore can be encodedwith little coding error. The interframe differential signal 404 isencoded in the first stage vector quantization coding section 405. Thenthe threshold value 422 is used as parameter. The coded data 406 encodedin the first stage vector quantization coding section 405 is decoded inthe first stage vector quantization decoding section 407; thereby thefirst stage decoded signal 408 is obtained. The first stage decodedsignal 408 is subtracted from the interframe differential signal 404 bythe subtractor 409; thereby the first stage error signal 410 isobtained. The first stage error signal 410 has little power further incomparison to the interframe differential signal 404, and therefore canbe encoded with little coding error. The first stage error signal 410 isencoded in the second stage vector quantization coding section 411. Thenthe threshold value 422 is used as parameter. The coded data 412 encodedin the second stage vector quantization coding section 411 is decoded inthe second stage vector quantization decoding section 413; thereby thesecond stage decoded signal 414 is obtained. The first stage decodedsignal 408 and the second stage decoded signal 414 are added in theadder 415; thereby the decoded interframe differential signal 416 isobtained. The interframe predicting signal 403 and the decodedinterframe differential signal 416 are added in the adder 417; therebythe decoded image signal series 418 is obtained. The decoded imagesignal series 418 is stored temporarily in the frame memory 419 andsupplied with the frame delay; thereby the interframe predicting signalis formed. On the other hand, the first stage coded data 406 and thesecond stage coded data 412 are subjected to the variable length codingin the variable length coding section 420, and stored temporarily in thebuffer 421 and subjected to the speed smoothing processing, and thentransmitted as the transmitting signal 424 through the line interface(I/F) 423. In the buffer 421, the threshold value 422 in proportion tothe storage amount of the variable length coded data is outputted, andsupplied to the first stage vector quantization coding section 405 andthe second stage vector quantization coding section 411 so as to controlthe information generation amount.

Next, referring to FIG. 26 and FIG. 27, the operation of the first stagevector quantization coding section 405 and the second stage vectorquantization coding section 411 will be described. As describedregarding the interframe vector quantization device in the prior art, inthe vector quantization, the picture element is made a plurality ofblocks and the output vector with the highest similarity (leastdistortion) is searched. In FIG. 26, an example of the vector quantizedin the first vector coding section 455 and the vector quantized in thesecond stage vector quantization coding section 411 is shown. Thedimensions of the vector in any case are made 4×4=16. In the invention,the average value of interframe differential signals corresponding tothe same position on the picture plane as that of the vector quantizedat the second stage (formed by the first stage error signal) is made oneelement, and vector quantized at the first stage is formed.Consequently, as shown in FIG. 26, when the vector quantized at thefirst stage and the vector quantized at the second stage are overlaid onthe picture plane, one piece at the first stage corresponds to 16 piecesat the second stage. This is expressed by formulas as hereinafterdescribed. If the series comprising the interframe differential signals404 blocked per 4×4=16 pieces is represented as S_(i) =[S_(i1), S_(i2),. . . , S_(i16) ], and set comprising the system S_(i) blocked furtherper 4×4=16 pieces on the picture plane is represented as [S]=[S₁, S₂, .. . , S₁₆ ] (S_(i) ε[S], i=1˜16), FIG. 26 is expressed as follows:##EQU25## If the first stage decoded signals 408 obtained at the firststage vector quantization decoding section 405 and the first stagevector quantization section 407 are represented as m'=[m₁ ', m₂ ', . . ., m₁₆ '], the first stage error signals 410 to be subjected to thevector quantization at the second stage are the following vectors:##EQU26## At the second stage, the above-mentioned vectors are subjectedto the average value separation and normalization, and then to thevector quantization as described regarding the interframe vectorquantization device in the prior art. Consequently, influence of thefirst stage vector quantization to the second stage vector quantizationrelates only to the average value being the scalar quantity, and thereinno influence as vector. If there is any influence between both,efficiency of the vector quantization at the second stage isdeteriorated by error of the vector quantization at the first stage. Ifrelation between the first stage and the second stage exists only in thescalar quantity as in the case of the invention, the first stage vectorquantization does not adversely affect the second stage vectorquantization.

FIG. 27 is a block diagram illustrating composition of the first stagevector quantization coding section 405. Input signal to be subjected tothe vector quantization is the interframe differential signal 404. Theaverage value and the amplitude of the signal 404 in unit of a block of4×4=16 pieces calculated in the operation section 425 of average valueand amplitude. This block has amount of a block to be subjected to thevector quantization at the second stage. The obtained average value 426is blocked, and the vector for the first stage quantization is formedand subjected to the average value separation and normalization in theaverage value separation and normalization section 428. This isexpressed by formulas as hereinafter described. The operation in theoperation section 425 of average value and amplitude is to calculate##EQU27## The processing in the average value separation andnormalization section 428 is to calculate ##EQU28## Distortion betweenthe normalized vectors X=[x₁, x₂, . . . , x₁₆ ] obtained as abovedescribed and the output vectors 432 read from the code book iscalculated in the distortion operation section 433. Before thisprocessing, the average value m_(i) and the amplitude g_(i) (427) arecompared with the threshold value 422 in the block discriminationsection 434. Let the threshold value 422 be Th, the block discriminationis represented as follows: ##EQU29## Regarding the insignificant block,the coding is not required at the first stage and the second stage.However, since the block discrimination uses the block in the secondvector quantization coding device 411 as unit, it is only one elementfor the block of the first stage. Consequently, in the distortionoperation section 433, the distortion of the element corresponding tothe insignificant block is estimated as 0. That is, the distortion 436is usually calculated as follows: ##EQU30## However, if the block S_(i)is insignificant block, since the corresponding element becomes S_(i)→m_(i) →x_(i), the distortion 436 is calculated as follows: ##EQU31## Inthe minimum distortion detecting section 437, among the distortion 436between the input vector stored in the code book 431 and the inputvector 429, the minimum distortion is detected and the index number 438of the input vector giving the minimum distortion is outputted. Thecoded data 406 outputted from the first stage vector quantization codingsection 405 are the block discrimination information 435, the averagevalue M and the amplitude G (430), and the index 438. However, if allblocks included in the blocks to be quantized at the first stage areinsignificant blocks, since information other than the blockdiscrimination information 435 does not have any meaning and need not betransmitted, the information amount can be controlled by the thresholdvalue 422.

Regarding the second stage vector quantization coding section 411, theoperation is similar to that of the vector quantization coding section439 described regarding the interframe vector quantization device in theprior art. In the embodiment, since the threshold value 422 supplied tothe second vector quantization coding section 411 may be controlled andonly the first stage vector quantization coding section 405 may be usedfor the coding, the information amount can be significantly controlled.

In this embodiment, although the control of the threshold value isfeedback control corresponding to the storage amount of the buffer, itis also effective that the frame memory is installed at the front stageof the coding device and the motion amount is estimated; thereby factorof feedforward is added.

Also in this embodiment, although the fixed output vector set is used inthe vector quantization code book at both the first stage and the secondstage, it is also effective that the code book for the first stageproduces the output vector using frame memory content within the loopand the method of vector quantization or motion compensation is used.Merit in that the average value of the block to perform coding of thesecond stage is treated as the picture element in the first stage is notlimited to the vector quantization, but, for example, use of theorthogonal transformation in coding of the second stage is alsoeffective because higher harmonics are not generated due to error in thefirst stage.

A sixth embodiment of the invention will be described referring to FIG.30-FIG. 32. In FIG. 30, numeral 528 designates a dynamic code book RAMin which writing or reading is possible at any time, numeral 529designates a normalization circuit which normalizes the average valueseparated input vector X in magnitude X of the vector, numeral 530designates normalized coefficient hereinafter referred to as "σ", andnumeral 531 designates normalized input vector hereinafter referred toas "X*". Numeral 532 designates a dynamic code book control sectionwhich controls the update procedure of the dynamic code book RAM,numeral 533 designates write demand signals, numeral 534 designatesnormalized input vector with index, numerals 535a tl , 535b designateselectors, and numeral 536 designates vector data. In FIG. 31, numeral537 designates a vector data decoding section.

The vector quantization coding operation in an embodiment of theinvention will be described referring to FIG. 30. Input signal vector Sdesignated by numeral in FIG. 30 is converted by the average valueseparate circuit 502 into average value separated input vector Xdesignated by numeral 505 in FIG. 30, and then inputted to the innerproduct operation section 506 and the normalization circuit 529. Theaverage value is processed and transmitted in similar manner to theprior art. The inner product P (x, y_(i)) between the normalized outputvector y_(i) which is read from the fixed code book ROM 507 and thedynamic code book RAM 528 and designated by numeral 509 in FIG. 30 andthe average value separated input vector x is calculated in the innerproduct operation section 506 in similar manner to the prior art, ' andthen the normalized output vector y_(i) giving the maximum inner productvalue to the average value separated input vector x is detected, and theaddress signal 508 indicating address of the detected normalized outputvector y_(i) on the fixed code book ROM or the dynamic code book RAM isoutputted as the index 519 from the index latch 516. The maximum innerproduct value P_(max) is outputted as the correction amplitude gdesignated by numeral 513 in FIG. 30 from the maximum inner productdetecting section 512. In the above-mentioned process, the inner productoperation section 506, the maximum inner product detecting section 512,the address counter 510 and the index latch 516 execute the sameoperation as that of the prior art. The dynamic code book RAM 528 asshown in FIG. 32 is constituted by a FIFO memory in first-in first-outform where writing or reading operation is performed in asynchronousstate.

Update is performed at any time according to the update procedurecontrolled by the dynamic code book control section 532. An example ofthe update procedure of the content of the dynamic code book RAM 528will now be described based on an operation example of the dynamic codebook control section 532. The average value separated input vector x isinputted to the normalization circuit 529 and processed according to thefollowing formulas, and analyzed into the normalization coefficient σdesignated by numeral 530 in FIG. 30 and the normalized input vector X*designated by numeral 531 in FIG. 30, and then two parameters σ and X*are supplied to the dynamic code book control section 532. ##EQU32##

In the dynamic code book control section 532, using the normalizedcoefficient σ and the correction amplitude g, the approximation d²between the average value separated input vector x and the normalizedoutput vector g·y_(i) regenerated in amplitude at the decoded state isestimated according to the following formula, and the update procedureof the dynamic code book RAM 528 is adaptively controlled based on themagnitude of the approximation d². ##EQU33## That is, when theapproximation d² is larger than the allowable value D², the write demandsignal 533 is made |1| and the normalized input vector X* is written inthe dynamic code book RAM 528, and at the same time the normalized inputvector 534 with index to which prefix is added so as to indicate thewriting of the normalized input vector X* is outputted. On the otherhand, when the approximation d² is the allowable value D² or less, thewrite demand signal 538 is made |0| and simply outputted. The writecontrol signal 533 is transmitted to the third selector 535c, and usedto select the coded data transmitted corresponding to the updateprocedure of the dynamic code book RAM 528. That is, when the writedemand signal 533 is |0|, since the update of the dynamic code book RAM528 is not performed, the first selector supplies the correctionamplitude g (numeral 513) outputted from the maximum inner productdetecting section 512 to the amplitude coding device 515 in similarmanner to the prior art, and the second selector 535b transmits theindex 519 outputted from the index latch 516 as the vector data 536 tothe coding data multiplexing section 520. When the write demand signal533 is |1|, the update of the dynamic code RAM is performed also at thereceiving side, and at the same time the normalized output vector y_(i)at the decoded state is replaced by the normalized input vector X*(numeral 531), and in order to transmit the normalized coefficient σ(numeral 530) obtained in the normalization processing, the firstselector 535a supplies the normalized coefficient σ to the amplitudecoding device 515 and the second selector 535b supplies the normalizedinput vector 534 with index as the vector data 536 to the coding datamultiplexing section 520. The amplitude coding device 515 performssimilar operation to the prior art.

Through the above-mentioned processing, the average value coded data517, the amplitude coded data 518 and the vector data 536 are multipliedin the coded data multiplexing section 520 according to the prescribedformat, and then outputted as the decoded output data 521.

An operation example of the dynamic code book RAM 528 will be describedreferring to FIG. 32. The read control section 528d reads data on theaddress assigned by the address pointer, i.e., the normalized outputvector y*, and repeats the operation in sequence.

When the write demand signal 535 is |1|, the write control section 528bwrites the write data, i.e., the normalized input vector X* onto theaddress assigned by the write address pointer. The address counter 528acounts up the address by 1 at the write finishing point and resets theaddress to 0 when it exceeds the maximum value, and repeats theoperation. When the above-mentioned operation is executed, the newestnormalized input vector X* of the definite number determined by thememory capacity can be used as the normalized output vector y_(i) in thevector quantization.

The vector quantization decoding operation in this embodiment will bedescribed referring to FIG. 31. The coded output data 521 are separatedin the coded data demultiplexing section 522 according to the prescribedformat into the average value coded data 517, the amplitude coded data518 and the vector data 536.

The average value coded data 517 and the amplitude coded data 518 aredecoded in similar manner to the prior art, and converted into theaverage value decoded value μ(numeral 525 in FIG. 31) and the amplitudedecoded value g (numeral 526 in FIG. 31). In the vector data decodingsection, it is discriminated whether the separated vector data 536 iscoincident with the prefix added to the normalization input vector ornot. If coincident, the write demand signal 533 is made |1|, and thewrite demand signal 533 and the normalized input vector X* (numeral 531in FIG. 31) received subsequent to the prefix are outputted. If notcoincident, the write demand signal 533 is made |0| and the vector data536 is outputted as the index 519. The fixed code book ROM 507 and thedynamic code book RAM 528 supply the normalized output vector y_(i)(numeral 509 in FIG. 31) corresponding to the index 519 to the thirdselector 535c. Further when the write demand signal 533 is |1|, thenormalized input vector X* (numeral 531 in FIG. 31) is written in thedynamic code book RAM 528, and the update of the dynamic code book RAM528 similar to the coding operation is executed. In the third selector535c, when the write demand signal 533 is |0| the normalized outputvector y_(i) (numeral 509 in FIG. 31) is outputted, and when the writedemand signal 533 is |1| the normalized input vector x (numeral 525 inFIG. 31) is outputted. The vector outputted from the third selector 535cis multiplied by the amplitude decoded value 526, and then the averagevalue decoded value 525 is added, thereby the decoded vector s (numeral527 in FIG. 31) is obtained.

In this embodiment, although the average value separation andnormalization vector quantization device is shown where the averagevalue of the input vector is separated and the output vector isseparated in the average value and normalized, similar effect can beobtained when the input vector is subjected to vector quantizationdirectly, and the dynamic code book RAM is updated based on the vectorquantization error.

A seventh embodiment of the invention will be described referring toFIG. 35.

FIG. 35 is a block constitution diagram of an image coding transmissionsystem according to the invention. In FIG. 35, the side ○A indicates thetransmitting side, and the side ○B indicates the receiving side.

In FIG. 35, numeral 601 designates an input buffer which inputs thedigitized image signals and outputs them suitably to the coding sectionat the next stage, numeral 603 designates a frame memory which storesthe image signals after coding and decoding before the present imagesignals by one frame, numeral 602 designates a subtractor which carriesout the subtraction between the output of the input buffer 601 and theoutput of the frame memory 603, numeral 604 designates a quantizationcoding device which gives the quantization and coding to the output ofthe frame memory 603, numeral 605 designates a quantization decodingdevice which decodes the signals after the quantization coding, numeral606 designates an interframe adder which adds the quantization decodedoutput and the output of the frame memory 603, and writes the result tothe frame memory 603, numeral 607 designates a variable length codingdevice which assigns the variable length code to the quantization codedoutput corresponding to the frequency of each code, numeral 608designates a transmitting buffer which stores the variable length codedoutput, numeral 609 designates a transmitting buffer control sectionwhich monitors the control of writing and reading in the transmittingbuffer 608 and the storage amount of the transmitting buffer and thentransmits the monitoring result to the input buffer 601, numeral 610designates a dummy data adding section which adds the dummy data to theoutput of the transmitting buffer, numeral 616 designates a localdecoding capability information generation section which generatesinformation indicating time capability of decoding processing at thelocal receiving side, numeral 617 designates a multiplexing sectionwhich multiplies the output of the dummy adding section by the localdecoding capability information generated in the section 616, andnumeral 611 designates a line interface section.

Numeral 612 designates a line interface section on the receiving side,numeral 619 designates a separate section which separates remotedecoding capability information from the receiving data, numeral 618designates a transmitting operation control section which determines thelocal transmitting operation from the remote decoding capabilityinformation and transmits it to the transmitting side, numeral 613designates a dummy separate section which deletes the added dummy data,numeral 614 designates a variable length decoding section which decodesthe variable length code, numeral 615 designates a receiving bufferwhich stores the signals after the variable length decoding, numeral605' designates a quantization decoding section which gives thequantization decoding to the output of the receiving buffer 615, numeral603' designates a frame memory which stores the decoded image signalsbefore the present image signals by one frame, and numeral 606'designates an interframe adder which adds the output of the quantizationdecoding section 605' and the output of the frame memory 603' and thenwrites the result to the frame memory 603'.

The operation of the device will be described.

The inputted image signals 701 are written to the input buffer 601. Theinput buffer performs writing and reading by the unit of the imageframe, but it has the composition of double buffer because reading maybe performed during writing.

The coded and decoded image signals 702 before the present image signalsby one frame are outputted from the frame memory 603. In the interframesubtractor 602, the interframe differential signals 703 are obtained bysubtracting between the present image signals 701' read from the imagebuffer and the image signals 702. The interframe differential signals703 are encoded by the quantization coding device 604, and become thequantization coded signals 704. FIG. 37 shows an example ofcharacteristics of the quantization coding device. The quantizationcoded signals 704 are inputted in the variable length coding device 607,and transmitted into the variable length code 706 corresponding to thefrequency of each coding signal.

At the same time, the quantization coded signals 704 are inputted in thequantization decoding device 605, and then are outputted as the codedand decoded differential signals 705. FIG. 38 shows an example ofcharacteristics of the quantization decoding device.

The coded and decoded differential signals 705 are inputted togetherwith the image signals 702 into the interframe adder 606, and become thecoded and decoded image signals 702 and are written to the frame memory603 for the coding to the next frame.

On the other hand, the variable length codes 706 are inputted in thetransmitting buffer 608. The transmitting buffer outputs the data inaccordance with the requirement from the transmission line side afterstoring the variable length codes over the definite amount, and has thecomposition of double buffer (buffer #1, buffer #2) because writing andreading must be performed at the same time. The transmitting buffercontrol section 609 controls writing and reading of the transmittingbuffer. For example, when the buffer #1 is at writing operation and thebuffer #2 is at reading operation, the transmitting buffer controlsection 609 monitors the storage amount of the buffer #1, and if thestorage amount becomes more than the prescribed set value, thetransmitting buffer control section 609 demands ceasing of output of thedata to the input buffer 601.

Receiving the demands, the input buffer 601 ceases the output of thedata to the rear stage. The transmitting buffer control section 609detects the pause of the input data to the transmitting buffer 608, andceases writing to the buffer #1 and makes the situation of waiting forreading. The buffer #2 during reading ceases reading if the residualamount becomes less than the prescribed set value, and it waits for thebuffer #1 to be in the situation of waiting for reading. When the buffer#1 is in the situation of waiting for reading, the buffer #2 and thebuffer #1 are read out continuously. The buffer #2 is in the situationof waiting for writing when the residual amount becomes zero.

When the buffer #2 is in the situation of waiting for writing, thetransmitting buffer control section 609 demands to start the output ofthe data to the input buffer 601.

In this process, before the buffer #1 gets in the situation of waitingfor reading, there occurs the situation that the transmitting buffer 608cannot output any data.

The dummy data adding section 610 outputs the data with the dummy dataadded thereto so as to continue the transmission of the data to thetransmission line without break while the transmitting buffer 608 cannotoutput the data.

In the local decoding capability information generation section, theinformation 707 indicating time capability of decoding processing atlocal receiving side, for example, information indicating time T_(D) 'required for processing one image frame is generated.

This information is multiplied in the multiplexing section 617 into thatcombined with the image dummy data. The signal after multiplexing isconverted in the line interface section 611 so that the electric levelis matched to characteristics of the transmission line, and thenoutputted to the transmission line.

At the receiving side, signal inputted through the transmission line issubjected to reverse conversion of the electric level in the lineinterface section 612, and information 707' indicating time capabilityof the remote decoding processing is separated from the signal in theseparate section 619 and transmitted to the transmitting operationcontrol section 618, and other data are transmitted to the dummyseparate section 613. In the dummy separate section 613, the dummy dataadded in the dummy adding section 610 are cleared; thereby only the dataabout the image are outputted.

The output is processed in the variable length decoding section 614 bythe reverse treatment with respect to that in the variable length codingsection 607, and then inputted in the form of the quantization codedsignals into the receiving buffer 615.

The receiving buffer has the composition of double buffer, becausewriting and reading are performed at the same time. In the receivingbuffer, the stored data are variable in amount so as to take matchingwith respect to time between the signal speed inputted from thetransmission line side and the speed of the image decoding section atthe rear stage.

For example, if the processing speed of the image decoding section atthe next stage is low, the stored amount of the receiving buffer isincreased. On the contrary, if the processing speed is high, thereceiving buffer acts at the small stored amount. The quantization codedsignals 704' are decoded by the quantization decoding device 605' andoutputted as the coded and decoded differential signals 705' in similarmanner to the transmitting side. The coded and decoded image signals702' before the present decoding image by one frame are outputted fromthe frame memory 603', and are added to the coded and decodeddifferential signals 705' in the interframe adder 606', and theresulting signals are written as the coded and decoded image signals tothe frame memory 603' and also outputted to outside.

On the other hand, at the transmitting operation control section 618,based on the information 707', discrimination is performed regardingwhat degree of the coding capability at the local transmitting sideenables the decoding processing without overflowing of the receivingbuffer at the remote station.

For example, if time T_(D) ' is required to decode one frame as theinformation 707', command is issued so that the interval to transmit thecode corresponding to the lead of each image frame from the transmittingbuffer is always made T_(D) ' or more.

Operation of the transmitting buffer control section 609 receiving thecode will now be described in detail.

FIG. 39 shows state of the transmitting buffer control section typicallywhen the buffer #1 is at reading state and the buffer #2 finishes thewriting and is waiting for reading. Usually, if the transmitting bufferstores data more than the prescribed set value (Th2 in FIG. 39) duringwriting, stoppage of data output is demanded to the input buffer 601. InFIG. 39, the buffer #2 stops the writing at the storage amount M.

On the other hand, reading from the transmitting buffer is performed ata burst in the N data unit, and when the residual amount of the bufferduring reading becomes less than N, reading is performed only the bufferat reverse side is at waiting for reading, and reading is stopped atother cases.

When command is issued from the transmitting operation control section618 that transmitting interval of the code corresponding to the lead ofthe image frame is made T_(D) ' or more, output of data corresponding tothe lead of the image frame from the transmitting buffer is checked.Once the corresponding data is outputted, the timer is set to the timeT_(D) ' and supervision is performed so that next corresponding data isnot outputted before the timer becomes time out.

If it is recognized that the corresponding data exists in the N data tobe subsequently transmitted before the time outs next reading isinhibited.

The longer the read inhibiting time, the longer the time of output stopcommand for data to the input buffer 601. Consequently, the image framenumber which can be encoded per unit time is decreased.

Since the dummy data are transmitted to the transmission line duringread inhibiting of the transmitting buffer, data of only image after thedummy data separation at the receiving side is intermittent in time,thereby longer time can be used for the decoding processing at thereceiving side.

Consequently, even at the image coding device having the receiving sideof large T_(D) ', since transmission at the remote transmitting side isperformed in matching with the local capability (limiting the capabilityat the transmitting side), although the image frame number is small andquality as the motion picture image is deteriorated, the device at thereceiving side is made of small size and at low cost.

In this embodiment, in order that the transmission image frame number atthe transmitting side is drawn to the limit of the capability at thereceiving side, output of the data corresponding to the lead of theimage from the transmitting buffer is limited. However, if the operationis allowed at considerably lower level than the capability at thereceiving side, above-mentioned effect can be obtained when the imageframe number outputted from the input buffer 601 per unit time isvaried.

Also in this embodiment, although the information 707 indicating thetime capability of the decoding processing and the image data aremultiplied on one transmission line and transmitted, similar effect canbe obtained when the information 707 is transmitted through anothertransmission line.

According to the invention as above described, in the motioncompensation circuit, the average value of the input block signals isadopted in matching between blocks, thereby even at large motion amountor at scene change, the frequency error signal is suppressed withoutincreasing the hardware scale or the processing time. As a result, thegenerated information amount is reduced or the picture quality isimproved.

Also according to the invention, since the ON/OFF operation of eachcoding and decoding device is adaptively selected and changed, thegenerating amount of the coding information and the reconstructed imagequality can be adaptively controlled per frame unit in wide range.

Further according to the invention, since the motion amount of the frameto be encoded hereafter is estimated and the coding control isperformed, if the subject is suddenly moved or stands still on thecontrary, the interframe coding device without delayed control isobtained.

Further according to the invention, the dimension number of the vectorsto be encoded is reduced and the vector quantization is performed instages. Still further, since the threshold processing is performed atthe encoding process of the second stage or later, the block size can bevaried without increasing the operation scale of the inner productquantization device, and control of the coded information amount in eachstage can be performed at wide range.

Further according to the invention, since the vector quantization deviceis constituted in multiple stages, vector quantization at the firststage is performed using the average value of blocks being encoding unitat the next stage as constitution element, control range of theinformation amount is wide, and the picture quality is not extremelydeteriorated by the control, and the vector quantization error at thefirst stage does not adversely affect the coding at the second stage.

Further according to the invention, since the code book of the vectorquantization device is composed of a dynamic code book where writing orreading is possible at any time and a fixed code book in theconventional manner, and based on approximation between the input vectorand the decoding vector, while content of the dynamic code book isadaptively updated, the vector quantization coding and decoding isexecuted, thereby deterioration of the approximation of the decodingvector can be suppressed with respect to special input vector beingdifferent in property from the output vector group stored in the fixedcode book.

Still further according to the invention, since the time capability ofthe decoding processing at the receiving side is transmitted togetherwith the image data to the remote station, and the time interval of datatransmission at the transmitting side is controlled based on theinformation at the receiving side, normal image communication can berealized even using the device of low cost and small size having lowdecoding processing capability at the receiving side.

What is claimed is:
 1. An average value predicting motion compensationoperator, wherein input digital image signals are divided into blocks ofprescribed size, and coding operation is performed to differentialsignals between the block of input signals and interframe predictingsignals produced block by block, a circuit for producing a block ofpredicting signals is constituted by input signal blocks and previousframe blocks disposed on the same position on the previous frame as theinput signal blocks, the blocks on the previous frame are givenaccording to L sorts (L: positive integer) of displacement (hereinafterreferred to as "motion vector"), and L blocks of the previous frame andaverage value of the input block is quantized into one of M levels (M:positive integer), further the quantized average value is formed as thevalue of picture element within the block and total (L+1) blocks aredefined as predicting signal blocks (hereinafter referred to as"reference block"), whereby the block pattern matching between the inputsignal block and the reference blocks is estimated by prescribedmeasurement method and the block being most matched with the inputsignal block is picked up from the reference blocks, and the picked upblock signal is outputted as the predicting signal block, and if thepicked up block is selected from L blocks of the previous frame blocks,the index of the picked up block corresponding to the motion vector isoutputted, and if the average value block is selected the index of thepicked up block corresponding to the quantized level of the averagevalue is outputted.
 2. An average value predicting motion compensationoperator as set forth in claim 1, comprising:a differential absolutevalue operator which performs pixel by pixel differential absolute valueprocess between a block of input signals and a block of signals disposedon the same position or near the position of the previous frame or ablock of samples with the quantized average value as hereinafterdescribed; an accumulator which performs distortion measure byaccumulation of the differential absolute value and average value foreach block by summarizing the input signals within the block; aquantizer which quantizes the average value into one of correspondinglevels of the matching distortion; a comparator which compares thematching distortion calculated with the accumulator between inputsignals and the signals on the same position or near the position onprevious frame or with the quantized average value, and outputs theblock of signals on the previous frame to give the minimum distortion orindex corresponding to the quantization average value; and a selectorwhich supplies the differential absolute value between the block ofinput signals and the blocks of signals on the previous frame or theblock of input signal to the accumulator.
 3. An average value predictingmotion compensation operator as set forth in claim 1 or claim 2, whereinin place of the differential absolute value operator, a differentialsquare value operator is used, and difference square value of eachpicture element is calculated between the input signals and thereference blocks.
 4. An average value predicting motion compensationoperator as set forth in claim 1 or claim 2, wherein in the accumulator,the matching distortion of each reference block is supplied withdifferent weighting functions thereby the specific reference block asthat giving the minimum distortion tends to be selected or not.
 5. Animage signal progressive build-up encoder/decoder wherein in case of oneinput image signal system, highly efficient coding and decoding isperformed, and obtained reconstructed image signal is delayed by oneframe, and the residual signal between the input image signal and thedelayed signal is again subjected to highly efficient coding anddecoding frame by frame and operation is repeatedly executed, therebyhigh quality images are achieved by progressive build-up in timeprogresses, said encoder/decoder comprising:an average valueencoder/decoder which calculates average value with a block of residualsignal vectors obtained by forming the block of the residual signal perlattice-shaped m×n (m, n: positive integer) sample, and performs highlyefficient coding of the average value; a normalized vector quantizationencoder/decoder which utilizes the residual signal vector as inputvector and subtracts the average value from each element of the inputvector and obtains the normalized output vector having the maximum innerproduct value to the input vector among a set of normalized outputvectors with average value 0, amplitude 1 generated so that total ofdistortion is minimized based on the distribution of normalized inputvectors of which amplitude becomes 1, and obtains an index to indicatethe normalized output vector, and also performs highly efficient codingand decoding of the amplitude gain of the input vector given by themaximum inner product value, and the decoded amplitude gain ismultiplied by the detected normalized output vector thereby theamplitude reconstructed output vector can be obtained; a sample valueencoder/decoder which performs highly efficient coding and decoding ofm×n block of samples in the residual signal vector sample by sample; aswitch which adaptively controls ON/OFF operation of the average valueencoder/decoder, the normalized vector quantization encoder/decoder andthe sample value encoder/decoder based on the control signal frame byframe; a vector reconstructor which selects adaptively the decodedaverage value, the decoded residual signal vector by adding the decodedaverage value to the amplitude reconstructed output vector, and thedecoded sample value based on the hereinafter described frame by framecontrol signal, and then outputs them as a decoded vector; and a codingcontroller which generates a control signal to enable the adaptivecontrol operation of the switch, each encoder/decoder and the vectorreconstructor corresponding to the number of coded samples each frame.6. An image signal progressive build-up encoder/decoder as set forth inclaim 5, wherein in the coding controller, at the first coding stage,only the average value encoder/decoder are turned on, and rough image isreconstructed by the decoded vector which all m×n samples aresubstituted by the decoded average value, and at the second codingstage, the average value encoder/decoder and the normalized vectorquantization encoder/decoder are turned on, and the decoded vectorsubstituted by the decoded residual signal vector is obtained, and theimage of which quality is better than that of the first stage can bebuilt up, and at the third coding stage or later, only the sample valueencoder/decoder is turned on, the decoding vector comprising thedecoding value of m×n pixels of sample value is obtained, and the imageof which quality is better than that of the second stage can be builtup.
 7. An image signal progressive build-up encoder/decoder as set forthin any of claim 5 or claim 6 wherein square-root of the mean squareerror of each sample between the residual signal vector and the decodedvector outputted from the vector reconstructor is calculated as theerror amplitude value, and based on the average of the error amplitudevalue during one frame period and the number of coding stage, the codingand decoding control mode and operation of the switch and the vectorreconstructor are adaptively selected and controlled frame by frame atthe subsequent coding stage.
 8. An image signal progressive build-upencoder/decoder as set forth in claim 5 or claim 6, wherein in theaverage value encoder/decoder, the normalized vector quantizationencoder/decoder and the sample value encoder/decoder, a set of differentcoding and decoding control modes corresponding to various encodedinformation generation rate of each coding stage are prepared, and basedon the total encoded information generation rate of one frame by framecoding stage and the number of the stage, the coding and decodingcontrol mode, operation of the switch and the vector reconstructor areadaptively selected and controlled frame by frame at the subsequentcoding stage.
 9. An image signal progressive build-up encoder/decoder asset forth in claim 7, wherein square-root of the mean square error ofeach sample between the residual signal vector and the decoded vectoroutputted from the vector reconstructor is calculated as the erroramplitude value, and based on the average of the error amplitude valueduring one frame period and the number of coding stage, the coding anddecoding control mode and operation of the switch and the vectorreconstructor are adaptively selected and controlled frame by frame atthe subsequent coding stage.
 10. An interframe motion image encoderwherein a first frame memory which can store motion image signals atleast by one frame is provided, interframe predicting signal is obtainedutilizing the first frame memory, and interframe prediction errorsignals are subjected to highly efficient coding and decoding and theencoded data are transmitted, and at the same time decoded image signalsare written in the first frame memory and updated for the interframeprediction of the subsequent frame or later, said encoder comprising:adiscriminator which evaluates approximation of the interframe predictingsignal to the input motion image signals to be coded, and then comparesthe approximation with a prescribed threshold value, and in the case ofthe large approximation, it generates the signal indicating that theinterframe predicting signals are treated as the decoded signal; anencoder and decoder which performs coding and decoding operation in thecase of a small approximation; a second frame memory which can store theinput motion image signals at least by one frame, and memorises theinput image signals before coding operation; an estimator which performssubstraction of the image signals of the previous frame read from thesecond frame memory and the input image signal, and then performsintegration frame by frame and obtains the estimation value of theinformation generation amount of the current frame; and a codingcontroller which judges the threshold value of the current coding framecorresponding to the corrected motion amount in order to smooth theinformation generation rate based on the method: the motion amount isestimated from the threshold value and the information generation amountof the previous coding frame, and then the motion amount is corrected bythe estimated information generation amount of the current coding frameoutputted from the estimator according to estimate the motion amount ofthe current coding frame, and the corrected motion amount is determined.11. An image signal progressive build-up encoder wherein to one inputimage frame, input signals are subjected to highly efficient coding anddecoding and reconstructed image signals are delayed by one frameperiod, and the residual signals between the input image signals and thedelayed signals are subjected to highly efficient coding and decodingframe by frame respectively and operation is repeatedly performed,thereby a high quality and resolution image is built up progressively astime progresses, said encoder comprising:a vector/subvector transformerwhich forms lattice-shaped am sample×bn line (a, b, m, n: natural numberand am=a×m, and bn=b×n) residual signals into an L-dimensional(L=am×bn)vector, and forms lattice-shaped m×n average values calculatedwithin the blocks of a sample×b line residual signals respectively intoa K-dimensional (K=m×n) subvector; an average value encoder and decoderwhich performs highly efficient coding and decoding of the arithmeticmean of the sample within the residual signal subvector; K-dimensionnormalized vector quantization encoder and decoder which treats theresidual signal subvector as K-dimension input vector, and performsvector quantization of the input vector through the inner productoperation of K dimensions into K-dimension normalized output vector ofaverage value 0 and amplitude 1, and performs highly efficient codingand decoding of the amplitude gain of the residual signal subvectorgiven by the inner product value between normalized output vector andthe input vector; a subvector/vector de-transformer which arrangessamples within the decoded K-dimension residual signal subvectorobtained by multiplying the normalized output vector and the decodedamplitude gain, and each element of the K-dimension amplitudereconstructed output vector corresponding to lattice-shaped m sample×nline subvector is added to the decoded average value, thereby performsinterpolation of the decoded residual signal vector having the Ldimensions; and decreases the size of the lattice-shaped block of theresidual signals step by step and to execute the coding and decodingoperation repeatedly.
 12. An image signal progressive build-up encoderas set forth in claim 11, wherein at the first coding stage, the inputimage signals are combined by lattice-shaped m sample×n line and averagevalue within the block is treated as one sample, and the samples arefurther combined in lattice-shaped m×n samples into K-dimension inputvector and the K-dimension mean separated and normalized vectorquantization is performed for the K-dimension input vector, and at thesubsequent coding stage, the K-dimension mean separated and normalizedvector quantization is performed again respectively by combination ofthe lattice-shaped m sample×n line the residual signals therebyhierarchical coding and decoding for each block can be performed.
 13. Animage signal progressive build-up encoder as set forth in claim 11 orclaim 12, wherein at the subsequent coding stage, the square mean of thesample value within the block by combination of the lattice-shaped msample×n line residual signals is compared with the threshold value, andblock by block conditional replenishment is executed for coding anddecoding of only the block having the square mean value larger than thethreshold value, and binary information indicating the result of thethreshold comparison is subjected to highly efficient coding anddecoding.
 14. An interframe vector quantizer in hierarchical multi-stagestructure, comprising:a frame memory which can store image signals of atleast one frame; a subtractor which subtracts interframe predictivesignals from input motion image signals to form interframe differentialsignals; average value and amplitude operation circuit which forms theinterframe differential signals obtained by the subtractor into theblocks with horizontal m picture elements by vertical n lines (m, nbeing positive integers) and calculates the first average value andamplitude; a block discrimination circuit of the first stage whichcompares the first average value and amplitude with the threshold valueof the first stage; an average value separation and normalizationcircuit of the first stage which forms the first average values intoblocks of horizontal m by vertical n corresponding spatially to m x nblocks of the interframe differential signals and calculates secondaverage value and amplitude and subtracts the second average value fromthe block of the first average values; a code table of the first stagewhich stores a set of output vectors suited for the distribution of theinput vector assuming that the block of the first average values is theinput vector of the first stage; a distortion operation circuit of thefirst stage which calculates distortion between the input vector of thefirst stage and the output vector; a minimum distortion detectingcircuit which detects the optimum output vector of the first stage whichgives the minimum distortion and determines the index of the optimumoutput vector of the first stage; a decoding circuit of the first stagewhich multiplies the optimum output vector of the first stage by thesecond amplitude and adds the second average value thereby obtains thedecoded vector of the first stage; a subtractor which subtracts thedecoded vector of the first stage from the interframe differentialsignal and obtains the error signal; an average value separation andnormalization circuit of the next stage which forms the error signalsinto the blocks with horizontal m elements by vertical n lines so thatthe blocks of the error signals corresponding spatially to the blocks ofthe interframe differential signals formed by the average value andamplitude operation circuit of the first-stage, and calculates the thirdaverage values and amplitude regarding blocks except for the block wherethe first average value and the amplitude are less than the thresholdvalue of the first stage, and subtracts the third average value from theblock of error signals and normalizes at the third amplitude; a blockdiscrimination circuit of the next stage which compares the thirdaverage value and amplitude with the threshold value of the next stage;a code table of the next stage which stores a set of output vectorssuited for the distribution of the input vectors of the next stageassuming that the blocks of the error signals except for ones where thethird average value and amplitude are less than the threshold value ofthe next stage in the block discrimination circuit of the second stage;a distortion operation circuit of the next stage which calculatesdistortion distance between the input vector of the next stage and theoutput vector; a minimum distortion detecting circuit of the next stagewhich detects the optimum output vector of the next stage which givesthe minimum distortion among the distortion calculated by the distortionoperation circuit of the next stage and determines the index of theoptimum output vector; a decoding circuit of the next stage whichmultiplies the optimum output vector of the next stage by the thirdamplitude and adds the third average value thereby obtains the decodedvector of the next stage; an adder which adds the decoded vector of thefirst stage and the decoded vector of the next stage and forms thedecoded interframe differential signals; an adder which adds the decodedinterframe differential signals to the interframe predictive signals andobtains the decoded image signal; a variable length coding section whichoperates variable length coding of the second and third average valueand amplitude and the index of the optimum output vector of the firststage and the next stage and the result of the block discriminationcircuit of the first stage and the next stage; and a buffer whichtemporarily stores data subjected to variable length coding and performssmoothing of the amount of information to be transmitted, wherein whenthe first average value and amplitude are detected to be less than thethreshold value of the first stage by the block discrimination circuitof the first stage, the decoded vector at the first stage and thecorresponding blocks of the next stage are assumed as zero vectors, whenthe third average value and amplitude are less than the threshold valueof the next stage the decoded vector of the next is assumed as zerovector, and threshold value at the first stage and the next stage ishierarchically controlled when the storage amount in the buffer issupposed as at least one of the control condition.
 15. An interframevector quantizer as set forth in claim 14, wherein when distortiondistance between input vector of the first stage and output vector readfrom the code book of the first stage is determined in the distortionoperation circuit of the first stage, element of the input vector of thefirst stage corresponding to the block having the first average valueand amplitude less than the threshold value in comparison in the blockdiscrimination circuit of the first stage is ignored in process of thedistortion calculation.
 16. An interframe vector quantizer as set forthin claim 14, wherein before the third average value and amplitude aresubjected to variable length coding in the variable length codingsection, a quantizer of average value and amplitude of subsequent stagehaving a plurality of quantization characteristics is previouslyprovided, and the plurality of quantization characteristics areadaptively changed by the second amplitude obtained in the average valueseparation and normalization circuit of the first stage.
 17. Aninterframe vector quantizer as set forth in claim 14, wherein pluralsets of output vectors having different characteristics are stored inthe code table of the next stage, and the plurality of quantizingcharacteristics are adaptively changed by the second amplitude obtainedin the average value separation and normalization circuit of the firststage.
 18. A dynamic vector quantizer provided with a coding sectioncomprising:an average value separation circuit which separates theaverage value within a block from input vectors formed with every ksamples (k being integer of two or more) of input signal; an averagevalue coding circuit which encodes the average value; a normalizationcircuit which normalizes the average value separated input vectorobtained through average value separation by the average valueseparation circuit according to the amplitude of the input vector; afixed code book which stores a plurality of normalized output vectorssuited for the distribution of the normalized input vectors; a dynamiccode book which stores the plurality of normalized input vectorssupposing them as normalized output vectors, the stored content beingupdated; an inner product operation section which calculates innerproduct value between the average value separated input vector and aplurality of normalized output vectors stored in the fixed code book andthe dynamic code book; a maximum inner product detecting section forfinding maximum inner product value among a plurality of inner productvalues calculated by the inner product operation section; an amplitudecoding circuit which encodes either the maximum inner product value orthe normalization coefficient; a dynamic code book control section whichcalculates a degree of approximation between the input vector and thedecoded vector after vector quantization of the input vector using themaximum inner product value and the normalization coefficient, andcontrols the updating operation of the dynamic code book based oncomparison result of the degree of approximation and threshold valuepreviously set; a coded data multiplexing section which multiplexeseither discrimination code relating to the update procedure, the outputdata of the average value coding circuit, the output data of theamplitude coding circuit and the maximum inner product value ornormalized output vector and the discrimination code is attachedaccording to the prescribed format, and then transmits it; means whichcalculates the degree of approximation with the normalizationcoefficient and the maximum inner product value and selects the maximuminner product value as input signal to the amplitude coding circuit whenthe degree of approximation is larger than the threshold value, anddefines the input signal to the coding data multiplexing section withthe discrimination code, of the output data of the average value codingcircuit, the output data of the amplitude coding circuit and thenormalized output vector; and means which replaces the oldest normalizedinput vector stored in the dynamic code book corresponding to the inputvector with the normalized input vector thereby executes the updateoperation when the degree of approximation is less than the thresholdvalue, and selects the normalization coefficient as input signal to theamplitude coding circuit, and further defines the input signal to thecoding data multiplexing section with the normalized input vector andthe output data of the average value coding circuit, the output data ofthe amplitude coding circuit and the discrimination symbol indicatingthat the updating operation has been done thereto, decoding sectioncomprising: a coded data demultiplexing section which separates thecoded data transmitted from the coding section into the average valuecoded data, the amplitude coded data and the discrimination symbolaccording to the prescribed format; an average value decoding circuitwhich decodes the average value coded data; an amplitude decodingcircuit which decodes the amplitude coded data; a fixed code book and adynamic code book corresponding to the fixed code book and the dynamiccode book at the side of the coding section respectively; means whichreads the normalized output vector on the address indicated by thediscrimination symbol from the fixed code book or the dynamic code bookwhen the discrimination symbol indicates the normalized output vector,and performs prescribed operation based on the normalized output vector,the decoded amplitude value outputted from the amplitude decodingcircuit, and the decoded average value outputted from the average valuedecoding circuit thereby obtains the decoded vector to the input vector;and means which replaces the oldest vector stored in the dynamic codebook with the normalized input vector received following thediscrimination symbol in the case that the discrimination symbolindicates the update operation thereby executes similar to the update inthe coding section, and performs prescribed operation of the normalizedinput vector based on the decoded amplitude value and decoded averagevalue thereby obtains the decoded vector to the input vector.
 19. Adynamic vector quantizer as set forth in claim 18, wherein in the codingsection, a plurality of normalized output vectors within the fixed codebook are arranged in tree structure in step per 2^(n) (n: naturalnumber), and a normalized output vector giving the maximum inner productvalue to the input vector is selected among 2^(n) pieces of thenormalized output vectors in sequence from upper step of the treestructure towards lower step and tree searching method is used.
 20. Adynamic vector quantizer as set forth in claim 18, wherein thenormalization circuit performs normalization using the root of the sumof the squared average value-separated-input vector as the normalizationcoefficient.
 21. A dynamic vector quantizer as set forth in claim 18,wherein the fixed code book stores a plurality of normalized outputvectors with average value zero and magnitude 1 determined depending onthe statistical characteristics of plural normalized input vectors. 22.A dynamic vector quantizer as set forth in claim 18, wherein the degreeof approximation between the input vector and the decoded vector aftervector quantization is estimated using the squared normalizationcoefficient and the squared maximum inner product value.
 23. A dynamicvector quantizer as set forth in claim 18, wherein the discriminationsymbol indicates the existence of the index corresponding to the outputvector, the normalized output vector is multiplied by the decodedamplitude value and the product is added to the decoded average value.24. A dynamic vector quantizer as set forth in claim 18, wherein thediscrimination symbol indicates that the update operation has been done,the normalized input vector is multiplied by the decoded amplitude valueand the product is added to the decoded average value.
 25. An imagecoding and decoding transmission system including at least two imagetransmitting/receiving stations, each station comprising:a transmitterincludingmeans for inputting a digitized image signal, means forencoding said digitized image signal, means for generating informationdata representing the amount of time required for a decoding process ofthe station, and means for multiplying and transmitting said encodeddigitized image signal and said information data; and a receiverincludingmeans for receiving multiplexed signals from a remote station,means for separating, from said multiplexed signals, information datarepresenting the amount of time required for a decoding process of saidremote station from coded image signal data, means for decoding saidcoded image signal data, and means for outputting said separatedinformation data to said transmitter to control the coding processingspeed to match the decoding processing time of said remote station. 26.An image coding and decoding transmission system as set forth in claim25, wherein based on the information indicating time requirement of thedecoding process of the remote station, the number of image frames to becoded per unit time in the transmitter may be controlled.
 27. An imagecoding and decoding transmission system as set forth in claim 25,wherein based on the information indicating time requirement of thedecoding process of the remote station, the number of image frames to betransmitted per unit time in the transmitter may be controlled.
 28. Animage coding and decoding transmission system as set forth in claim 25,wherein based on information indicating time requirement of the decodingprocess of the remote station, the minimum value of a time intervalbetween the transmitting of the code corresponding to the lead of eachimage frame at the transmitter may be controlled.